Purpose

This how-to guide shows how to integrate a Java  Messaging Services (JMS) Provider with SAP Data Services. This is a common Enterprise Application Integration scenario where a service is called asynchronously via request/response messages.  SAP Data Services’ role here is to provide a simple Real-Time service. Configuration includes quite a few steps to get everything up and running. This step-by-step configuration example covers all components that need to be touched including the JMS provider.

Overview

We want an external information resource (IR) – our JMS provider – to initiate a request by putting a request message into a request queue.  SAP Data Services is the JMS client that waits for request messages, executes a service and puts a correlated response message into a response queue.  We’re using the pre-built JMS adapter in SAP Data Services 4.2 and use Active MQ as JMS provider. Since we focus on Real-Time integration we’re not using an adapter datastore in this scenario. All incoming and outgoing data is received/sent back via messages. We will configure a Real-Time Job, check the settings of the Job Server and Access Server, configure a Real-Time Service, install Active MQ and configure the message queues, configure the JMS adapter and its operation and finally send test messages from the Active MQ console.

Real-Time Job
For our service we’re using a “Hello Word”-Real-Time Job named Job_TestConnectivity. For details, please refer to the  SAP Data Services 4.2 tutorial Chapter 14 . SAP Data Services comes with all the ATL, DTD and XML files in <DS_LINK_DIR>/ConnectivityTest to create  Job_TestConnectivity. The job reads an input message that has one input string…

…and returns an output message that has one output string with the first two words of the input string in reverse order:

Job Server

We need to make sure that one JobServer supports adapters. Using Data Services Server Manager utility, we switch on  “support adapter, message broker communication” and “Use SSL protocol for adapter, message broker communication”. We associate the Job Server with the repository that has the Real-Time Job Job_TestConnectivity. Finally we restart SAP Data Services by clicking “close and restart” or we restart it later using the Control Panel  => Administrative Tools => Services => SAP Data Services (right mouse click) => restart.

Access Server

We need to have an Access Server up and running. The Access Server  will receive the input messages from the JMS adapter and dispatch them to an instance of the Real-Time Service RS_TestConnectivity. In SAP Data Services Management Console choose Administrator => Management => Access Server and check if an Access Server is configured and add one if necessary. By default, the AccessServer uses port 4000.

Real-Time Service

We configure a Real-Time Service “RS_TestConnectivity” for our Real-Time Job Job_TestConnectivity. In SAP Data Services Management Console navigate to Administrator => Real-Time => <hostname>:4000 => Real-Time Services => Real-Time Service Configuration. Configure a new Real-Time Service “RS_TestConnectivity” and select Job_TestConnectivity with the Browse-Button:

Add the JobServer as Service Provider and click “Apply”. Start the Real-Time Service via Administrator => Real-Time => <hostname>:4000 => Real-Time Services => Real-Time Service Status, and click “Start”:

Active MQ – Installation

We could use any JMS provider but in this case we’re using Active MQ since it can be quickly installed and configured. Download and unzip Active MQ from http://activemq.apache.org/. In this scenario we use version 5.9.0 and we install it in C:\local\ActiveMQ on the same machine as SAP Data Services. At the command line change to directory C:\local\ActiveMQ\bin and execute activemq.bat:

Active MQ console

Now, we have our JMS provider up and running and we can access the Active MQ console at http://<hostname>:8161/admin . We’re using admin / admin to login.

The browser should now display the homepage of the Active MQ console:

We click on the “Queues” menu to add 3 queues named “FailedQueue”, “RequestQueue” and “ResponseQueue”:

Active MQ JMS client

The SAP Data Services JMS Adapter will access the JMS client provided by Active MQ to communicate with the JMS provider. The JMS client is in activemq-all-5.9.0.jar. We will add this jar file to the ClassPath of the JMS adapter later. According to the JNDI documentation of Active MQ we need to create a jndi.properties file and either add it to the ClassPath or put it into activemq-all-5.9.0.jar. The jndi.properties file maps the JNDI names of the queues to their physical names. Create jndi.properties as shown below. You can add it to activemq-all-5.9.0.jar e.g. by using WinZip.

JMS Adapter

Now we are ready to configure our JMS Adapter in SAP Data Services. In SAP Data Services Management Console, choose Administrator => Adapter Instances => Adapter Configuration…

Choose JMSAdapter…

Enter the configuration information as shown below:

  • set the adapter name; here: MyJMSAdapter
  • set the Access Server hostname and port; here: localhost, 4000

Remove the default entry of the ClassPath and add the following files to the ClassPath. All necessary jar files – except the JMS client jar file – are located in <DS_LINK_DIR>\lib\ or <DS_LINK_DIR>\ext\lib\. Replace <DS_LINK_DIR> with the respective directory of your installation.

  • <DS_LINK_DIR>\lib\acta_adapter_sdk.jar
  • <DS_LINK_DIR>\lib\acta_broker_client.jar
  • <DS_LINK_DIR>\lib\acta_jms_adapter.jar
  • <DS_LINK_DIR>\lib\acta_tool.jar
  • <DS_LINK_DIR>\ext\lib\ssljFIPS.jar
  • <DS_LINK_DIR>\ext\lib\cryptojFIPS.jar
  • <DS_LINK_DIR>\ext\lib\bcm.jar
  • <DS_LINK_DIR>\ext\lib\xercesImpl.jar
  • C:\local\ActiveMQ\activemq-all-5.9.0.jar (make sure it contains jndi.properties)

Note: The template file JMSadapter.xml that has the default ClassPath and all other default values and choices, is located in <DS_COMMON_DIR>\adapters\config\templates. You might want to adjust this file to have other defaults when configuring a new JMS adapter. Once an adapter is configured you need to change its configuration file located in <DS_COMMON_DIR>\adapters\config. On Windows <DS_COMMON_DIR> is %ALLUSERSPROFILE%\SAP BusinessObjects\Data Services by default.

JMS Adapter – JNDI configuration

We use the Java Naming and Directory Interface (JNDI)  to configure the JMS adapter. So we chose:

 

   Configuration Type:  JNDI

Next we set the Active MQ JNDI Name Server URL:

     Server URL: tcp://localhost:61616

For Active MQ we need to set the JNDI context factory to org.apache.activemq.jndi.ActiveMQInitialContextFactory (see ActiveMQ documentation section JNDI support). By default this string is not offered in the drop down box in the JNDI configuration section, so we need to edit <DS_COMMON_DIR>\adapters\config\templates\JMSAdapter.xml and add the string to the pipe-delimited list in the jndiFactory entry.

Note: If MyJMSAdapter already exists, we need to edit <DS_COMMON_DIR>\adapters\config\MyJMSAdapter.xml instead.

     <jndiFactory Choices=”org.apache.activemq.jndi.ActiveMQInitialContextFactory|     …   >

After navigating to Administrator => Adapter Instances => Adapter Instances  => My JMSAdapter  we choose the right string from the drop-down-list and set:

     Initial Naming Factory: org.apache.activemq.jndi.ActiveMQInitialContextFactory

Finally we set the Queue Connection Factory and Topic Connection Factory as described in the Active MQ documentation:

     Queue Connection Factory: QueueConnectionFactory

     Topic Connection Factory: TopicConnectionFactory

Click “Apply” to save all settings.

JMS Adapter – start

We are ready to check if the JMS adapter starts now. We still have to configure an operation for the adapter yet (see below) but we want to check first if our configuration works fine. There are many reasons that the adapter doesn’t start at first  – e.g. missing or wrong files in ClassPath, typos in JNDI configuration, etc. You will not find any entry in the error file and trace file in this case – these files are for messages created by the adapter when it is up and running. To find the reason for the adapter in case it doesn’t start, switch on Trace Mode=True in the JMS Adapter Configuration and restart the JMS Adapter. Then Check the Job Server’s log file in <DS_COMMON_DIR>/log/<jobserver>.log. Search for the java call the Job Server executes to launch the JMS Adapter. Copy the whole command, execute it from the command line and try to fix the problem by adjusting the command.If the JMS adapter starts properly the Adapter Instance Status will look like this:

JMS Adapter – operation configuration

Now we need to configure the  JMS operation. We’ll configure an operation of type “Get: Request/Reply” since we want our adapter to dequeue requests from the RequestQueue, pass it to the Real-Time Service, wait for the response and enqueue the response into the ResponseQueue.In DS Management Console navigate to Administrator =>  Adapter Instances => <jobserver>@<hostname>:<port> => Adapter Configuration => MyJMSAdapter => Operations and click “Add”. Select Operation Type “Get: Request/Reply and Request/Acknowledge using Queues” and click “Apply”.

Set the operation details as displayed below. Since our Real-Time Service will respond to requests quickly, we reduce the polling interval to 50 ms.

Click “Apply” to save the changes and restart the adapter. The adapter instance status should look like this:

JMS Adapter – test

Of course we want to see if the adapter works as expected. To do this we put a request message into the request queue and see what happens. We open the Active MQ console again (URL http://<hostname>:8161/admin, Login: admin/admin) and select “Send”. We create the request message as shown below:

After we have clicked “Send” the message is enqueued into RequestQueue, dequeued from there by the JMS adapter that passes the request to the Real-Time Service and receives the response from it. The JMS adapter finally puts the response message into ResponseQueue.

The Active MQ console should look like this after some seconds:

We have one message in the response queue. To display it we click on ResponseQueue and then on the message ID…

…and have a look at the repsonse message.  You should see the “World Hello” string in the message details.

To report this post you need to login first.

19 Comments

You must be Logged on to comment or reply to a post.

    1. Martin Bernhardt Post author

      Hi Subbarao,

      thanks!

      In Active MQ console you can view all the messages (request, response, failed).

      If you do not want the words to be twisted just change the mapping in the Real-Time Job Job_TestConnectivity as you like. The word-function is used there just to show that the message is processed by SAP Data Services at all.

      (0) 
      1. ROSHAN CHHAJED

        Hi Martin,

        I have a JMS adapter instance running fine. I need to connect WS MQ 7.5 with MQ and not JNDI.

        Can you explain a Put operation for MQ and how to set all parameters.

        I am confused about Message Format and Request XML Root Element

        (0) 
        1. Martin Bernhardt Post author

          Hi,

          you need to change MQ part of the adapter configuration (instead of JNDI)

          and the ClassPath must include the necessary WebSphere MQ classes to connect to WebSphere MQ.

          You can derive the root element from the XSD-file or DTD-file that describes the message format. In our case the root element would be test  (not TestOut as displayed in the last screenshot)

          (0) 
  1. Venkata Ramana Paidi

    Hi Martin,

    It is very nice explanation to integrate JMS.   I have followed all the steps  but I am getting error while see the response queue like

    Error!

    Exception occurred while processing this request, check the log for more information!

    What do you want to do next?


    My trace log details .




    I have seen the adapter error log but there is no entry for that duration.  I have checked with soapui real time service is working fine . What will be the issue?  Please help to fix this issue.

    Thanks & Regards,

    Ramana.

    (0) 
    1. Martin Bernhardt Post author

      Hi Ramana,

      sounds like RS_TestConnectivity cannot handle your Input XML.

      please check

      * in Designer: does Job_TestConnectivity work fine with your Input XML. (use your Input XML it as Test file in the dataflow)

      * the trace file of Job_TestConnectivity in <DS_COMMON_DIR>/log/<Jobserver>/<hostname>_<repo>_<dbuser> (e.g. file trace_11_12_2014_13_52_30_11__Job_TestConnectivity.txt)

      Regards, Martin

      (0) 
      1. Venkata Ramana Paidi

        Hi Martin,

        Thanks for your support.

        Job in the designer is working fine .  I have tested with SOAP UI also , It is working fine.

        I have changed the Polling interval time and timeout  as it is throwing timed out error in the MQActive.   Now it is showing same error in the ResponseQueue while it is opening .

        Error!

        Exception occurred while processing this request, check the log for more information!

        What do you want to do next?


        Adapter trace is log is as shown below.

        There is no information in the error log in for that duration.

        I have checked in the log folder  and there is no log file generated in data services when I am giving input from Requestqueue

        Please guede me to resolve this issue.

        Thanks & Regards,

        Ramana.

        (0) 
  2. Chandra K

    Hello Martin,

    Thanks for sharing the information on real-time processing using Data Services.

    We have implemented the same solution and wanted to know if the below scenario is possible.

    Can we write the same source xml message into the same queue if we find any data issues or referential integrity issues in the data?

    For example:

    – We have 2 tables Employee and Payrates with Employee_ID in both the tables.

    – There are 2 xmls generated from source database when there is a change on these 2 tables.

    – As the process to Insert/Update/Delete on these tables is real-time and the Employee record might be delayed due to some network or process issue, in this situation we don’t want to process the Payrates record as it can’t find the Employee ID in the Employee table. So, right now we are checking for the existence on the parent record and writing a xml file to a server location and processing them manually.

    – So, is there a way where we can write this message back to the same queue instead of writing it to the server as a file and processing manually.

    – This will help to remove the manual task and the system will create a loop by moving the file back to same location and process when the Employee record is processed.

    Please let me know if this is possible.

    Appreciate your help in advance.

    Thanks,

    Chandra

    (0) 
  3. Pradeep Shenoy

    Hello Martin,

    I am trying to connect Data Services with Tibco EMS. I followed all the steps mentioned in the blog along with all the necessary tibco related changes. But i am hitting a NullPointerException. My JMS Adapter is not recognizing the jndi config. i am getting the below error. Any idea, why?

    6/8/16 2:37:56 PM JMS adapter error: Error in getting the queue connection factory.: java.lang.NullPointerException: while trying to invoke the method com.acta.adapter.jms.JMSConfigHandler.getQueueConnectionFactory() of a null object loaded from field com.acta.adapter.jms.JMSAdapter.configHandler of an object loaded from local variable ‘this’

    Note- surprisingly i do not have the <DS_COMMON_DIR>/log/ or <DS_COMMON_DIR>/adapters/config folder

    Any inputs appreciated.

    (0) 
      1. Pradeep Shenoy

        Hi Martin,

        Thank you for your reply. Yes, I have checked my JNDI param config thoroghy. They are correct. I have also cross checked the JNDI param with a standalone Java JMS Client and it worked sucessfully.

        I looked at the blog you mentioned. I have Tibco EMS server running on my machine and the Queue and Topic are already created. So that should not be the problem.

        Finally i found the \adapters\config folder. It was under C:\ProgramData\SAP BusinessObjects\Data Services. Apparently that folder is hidden on my machine. Any idea Why? Is the folder also hidden on your machine? I checked the MyAdapter.xml file and the JNDI properties are correct.  Under the C:\ProgramData\SAP BusinessObjects\Data Services\adapters\log folder I found the trace and error log files. I found this error in the file.

        6/9/16 4:26:30 PM Attempt number 1 to connect to Access Server ‘localhost:4000’. Error=While sending a message to the socket ‘1b1fb339[SSL_NULL_WITH_NULL_NULL: Socket[addr=localhost/127.0.0.1,port=4000,localport=58535]]’ caught the exception: Connection reset. 6/9/16 4:26:47 PM [SAX Parser Error] :1:23 : cvc-elt.1: Cannot find the declaration of element ‘AdapterConfiguration’.


        I suspect it is either- An SSL related issue and the Access server is not accessible. OR The adapter is not able to find the ‘AdapterConfiguration’ file and hence hitting the NullPointer Exception.

        I think the adapter config file should be under C:\Program Files (x86)\SAP BusinessObjects\Data Services\adapters and NOT C:\ProgramData\SAP BusinessObjects\Data Services\adapters

        Do you agree? Please let me know your thoughts.!!

        Regards,

        Pradeep

        (0) 
        1. Martin Bernhardt Post author

          Hi Pradeep,

          the config files must definitely be where DS_COMMON_DIR points to, not where DS_LINK_DIR points to. DS_LINK_DIR has the templates with the default settings and choices you have – e.g. the list of JNDI context factory classes that is in the drop-down-list. So it’s correct that MyAdapter.xml is there.

          The second error message could mean there’s an issue with the structure of MyAdapter.xml (?).

          Regards, Martin

          (0) 
          1. Pradeep Shenoy

            Hi Martin,

            I tried a lot of permutations and combinations. But still its the same error-

            6/15/16 11:19:22 AM JMS adapter error: Error in getting the queue connection factory.: java.lang.NullPointerException: while trying to invoke the method com.acta.adapter.jms.JMSConfigHandler.getQueueConnectionFactory() of a null object loaded from field com.acta.adapter.jms.JMSAdapter.configHandler of an object loaded from local variable ‘this’

            The problem is that there is not much information in the log file either. I cannot track down the issue and pin point it to a particular place.

            Any other places I can check?

            BTW, Have you ever heard or come across a successful connection between SAP BODS and Tibco EMS? 🙂

            Regards,

            Pradeep

            (0) 
            1. Pradeep Shenoy

              Hi Martin,

              I finally got it working. The problem was with the Java version. (I dont know How or Why). I replaced the existing <%LINK_DIR%>/ext/jre with my local jre8 copy and restarted the Job server. Surprisingly the Adapter and the Operation started working.

              Thank you for your response. 🙂

              Thanks,

              Pradeep

              (0) 
    1. Pradeep Shenoy

      Hi @ Martin Bernhardt ,


      A step further from implementation of JMS Real-Time integration, Is it also possible to extract the metadata of the XML message which we read from the queue?

      As seen in your blog, when we see the message in the Response Queue through Active MQ console, we see the Message ID, Correlation ID and Update ID.

      How do we get these values inside Data Services?


      Best Regards,

      Pradeep Shenoy

      (0) 
      1. Martin Bernhardt Post author

        Hi Pradeep,

        I doubt that we have a chance to see this meta data in Data Services. These IDs are not passed to AccessServer / RealTime Service / dataflow.

        As a workaround you might want to put your own IDs into the XML structure and make it part of the payload.

        Regards, Martin

        (0) 

Leave a Reply