Continue of Bridges Construction Site (1/3): Asynchronous – synchronous bridge.

———————————————-

Hello again, colleagues!

Let’s continue to review solutions for the problem of integration in heterogeneous interfaces landscape using bridges.

Uff.. most complex part of this post is finished! 🙂

——————-

See also:

Bridges Construction Site (1/3): Asynchronous – synchronous bridge.

Bridges Construction Site (3/3): SAP PI bridges – “exotic” and recommendations.

——————-

/wp-content/uploads/2014/04/nbridge2_432432.jpg

3. Sync-Async Bridge and its modules – RequestOnewayBean, WaitResponseBean and NotifyResponseBean.

  Let’s think about a new problem: we are running the process in the one system and want to send some information to the external system. It’s simple. But we also need immediate confirmation of  of successful reception (or an error message).

Our external system works in asynchronous manner only, ie it takes the input by one service (it can be a file, asynchronous SOAP, SQL, HTTP, etc.), and send an acknowledgment by another service after while.

Let me remind you the scheme of such interfaces:

/wp-content/uploads/2014/04/bridge2_419376.jpg

Figure 1 : Synchronous – Asynchronous Bridge


Sync-async bridge using following modules:

RequestOnewayBean is responsible for converting synchronous message to asynchronous.

WaitResponseBean – leaves a synchronious communication channel open and waits for the response. When receiving asynchronous response – converts it to a synchronous message and sends it back to the original system. If the answer does not come within the specified time – sends an error message to the source system.

NotifyResponseBean is used instead of the standard adapter module in the asynchronous response sender channel to transfer the response message directly to WaitResponseBean ( bypassing the processing in the messaging subsystem of PI).

RequestOnewayBean and WaitResponseBean can be used in a sender communication channel or in a receiver communication channel.

RequestOnewayBean must be inserted before the standard adapter module (usually CallSAPAdapter); WaitResponseBean – after.

For using modules in the Communication Channel you should go to the tab «Module» and enter the following values:


Module name Type Module key
AF_Modules/RequestOnewayBean Local Enterprise Bean RqOneWay
AF_Modules/WaitResponseBean Local Enterprise Bean WResp
AF_Modules/NotifyResponseBean Local Enterprise Bean NResp


«Module Key» can be anything – so long as it is unique and the keys would be difficult to confuse – it will help us later to determine the parameters of modules.


Further settings depends on the placement of modules: in a sender communication channel or in a receiver communication channel.

But we have one more problem connected to synchronous-asynchronous bridge.

Imagine that several messages are going through the bridge simultaneously. So, at some point few queries are waiting for responses. Some response is coming from the target system – but how we know which of pending channels must be used to send a specific answer?

/wp-content/uploads/2014/04/bridge12_431861.jpg

Figure 2 : the task of determining the recipient of asynchronous response

Here comes new entity – a correlation. Correlation is an identifier , the “key” , which is held on the correspondence between the response and awaiting queue.

    The term ” correlation ” can also be found in the Sync-Async Bridge in ccBPM – there may be a correlation using not only one but a set of identifiers .

So the task is to “remember” some id during the sending of asynchronous message and check this id during the receiving an answer:/wp-content/uploads/2014/04/bridge11_431779.jpg

Figure 3: Determination of the recipient using the asynchronous response correlation

The correlationID parameter (defined in the PI message header) is used to correlate a response from asynchronous system. The value in correlationID must be equal to the message GUID of the synchronous request message . This parameter belongs to a group of Adapter-Specific Message Attributes and can be changed using the standard module – DynamicConfigurationBean.

Another example in SAP standard documentation («Configuring the Async / Sync Bridge Using the JMS Adapter») correlate messages using standard settings of communication channels JMS (“Set PI Conversation ID” parameter).

In other kinds of adapters similar standard parameter is missing and you have to make some tweaks – in particular , to save Message GUID and fill correlationID parameter using PI or with the help of the target system. In the example below (3.2) , you can find one of the ways to work with correlations in file adapter.

3.1 Synchronous-asynchronous bridge with modules in the Sender Communication Channel.

If we use Sender Communication Channel for bridge’s modules, the scheme of message processing would be next:

/wp-content/uploads/2014/04/bridge13_431891.jpg

Figure 4: The logic of the synchronous-asynchronous bridge with modules in the Sender CC.


  1. Receiving a synchronous message from external system via communication channel, sending it to the module RequestOnewayBean. The module switch the message type from synchronous to asynchronous by changing the message header.
  2. Depending on the parameter passThrough, module pass the message down the chain of modules or directly sending it to the message processing subsystem of PI (skipping step 3).
  3. Message processing by the standard adapter module, transfering it to PI message processing subsystem.
  4. Sending a request to asynchronous communication channel.
  5. Asynchronous call to the external system
  6. The response from external system is coming to async sender communication channel
  7. Return the response to the NotifyResponseBean, which directly transmits a response to WaitResponseBean, bypassing the messaging subsystem PI.
  8. WaitResponseBean changes message type from asynchronous to synchronous and returns it to the source system.

Module parameters for RequestOnewayBean when placing it in the Sender Communication Channel:


Parameter

Function

Possible values Default Values
passThrough

Specifies where module must pass the processed message :

– PI message processing subsystem – false

– Next module in the chain – true

true / false false


Module parameters for WaitResponseBean when placing it in the Sender Communication Channel:


Parameter Function Possible values Default Values
timeout Response waiting time in milliseconds 300000


NotifyResponseBean module in this configuration must be located in the recieving communication channel for a response from asynchronous system.

The module must be used instead of the standard adapter module.

NotifyResponseBean have following module parameters:

Parameter Function Possible Values Default Values
timeout Module waiting time in milliseconds 300000
fault Name of an error message For example, SA_BRIDGE_ERROR
faultNamespace Namespace of an error message:
http://sap.com/xi/XI/System — an error would be count as a system error;
any other namespace – an error would be counted as an application error.

Once we set up all the necessary modules , we need to configure the request and response correlation.

To do so, we need keep the message GUID somehow and pass it to the target system. The system, in turn, should return a response with a the GUID in some part of message. When the answer is coming, we need to fill out the parameter in the message header responsible for the correlation – correlationId (Dynamic Configuration header section) with stored GUID value.

Solution can be different for each of the adapter types. I’ll show you one of possible file adapter solution in the example below.

After setting up the correlation the only thing to do is to create routing rules linking source synchronous interface to the target asynchronous interface.

For the routing of response it is sufficient to create a Sender Agreement only – everything else will be complete by the NotifyResponseBean module.

3.2 Example of synchronous – asynchronous bridge with modules in Sender Communication Channel.

Let’s take a look to the specific example.

Business case: we have an external client who gives us an information about new books available in the store using a synchronous web-service .

PI should receive this information and send it to our database. This DB is pretty old, so the Interface of DB is based on the file exchange. The status of operation  is also returned as a file, so PI should get it and return it to the client.

/wp-content/uploads/2014/04/bridge14_432089.jpg

Figure 5: Example diagram.

To implement this interface, we need to create the following development objects:

/wp-content/uploads/2014/04/bridge15_432091.jpg

Figure 6: Development objects in the Integration Repository

Please note the operations interfaces mapping – it synchronous, but “one-way” – it works in one direction only. To create it we need to use a “trick” with a temporary changing of receiver interface type, like it was done in the mapping from the previous article about the asynchronous-synchronous bridges.

After that, create all necessary routing objects in the Integration Builder:

/wp-content/uploads/2014/04/bridge16_432120.jpg

Figure 7: Configuration objects

The question may arise – how the routing of response can be done by Sender Agreement only?

You can configure full routing rule for the answer or make response routing via Integrated Configuration. But in this particular case it does not make sense – the answer still will be transferred to the module NotifyResponseBean and directly to waiting module WaitResponseBean after that, bypassing the messages processing subsystem of PI.

Now let’s put the modules into place.

Synchronous SOAP communication channel:

/wp-content/uploads/2014/04/sapbridge_example_36_c_432121.jpg

Figure 8: Synchronous SOAP Sender communication channel

Please don’t forget to set Quality Of Service = Best Effort, ie identify the channel as synchronous.

Then add modules RequestOnewayBean and WaitResponseBean:

/wp-content/uploads/2014/04/sapbridge_example_37_c_432132.jpg

Fig. 9: Modules in SOAP Sender synchronous communication channel

Then set up a file communication channel to submit a request to the database:

/wp-content/uploads/2014/04/sapbridge_example_38_c_432133.jpg

Fig.: 10: Asynchronous File communication channel for DB request.

Please note the configuration of file name:

      Filename is formed by a variable that have Message ID value. Filename = message GUID without any extension.

    Why do so? Thus, in our example, we store the GUID of request message to use it in correlation later. In reality, you can do it same way. or you can pass GUID in the message body. The main thing – target system must get it, save it and return it in an accessible manner to PI along with the response.

Next step: set up a file communication channel for the response:

/wp-content/uploads/2014/04/sapbridge_example_39_c_432158.jpg

Fig. 11: File communication channel to get a response from the target system.

We assume that the system produces a response in another local directory on the PI-server as an XML-file.

The name of the file must be the same GUID as request.

Modules configuration:

/wp-content/uploads/2014/04/sapbridge_example_40_c_432159.jpg

Fig.: 12: Modules in asynchronous file sender comm. channel

Standard module DynamicConfigurationBean writes the file name to the correlationId parameter in the header of PI message – ie fills correlation for further processing of the response by the bridge.

We are using NotifyResponseBean module instead of standard adapter module CallSAPAdapter. NotifyResponseBean forwards the response directly to the waiting module WaitResponseBean. If nobody expects the answer with such correlation value – an error will be written to the log.

Well, that’s all settings now, time to check our bridge.

Form a WSDL for web-service (use context menu on Integrated Configuration -> Display WSDL). Then, use WSDL to form and send SOAP request using any suitable SOAP-tool (I used a freeware tool SOAPUI).

/wp-content/uploads/2014/04/sapbridge_example_41_c_432229.jpg

Fig.: 13: Sending a synchronous request from SOAPUI

After sending the file appears in the directory with the name of the message GUID and request data inside.

/wp-content/uploads/2014/04/sapbridge_example_43_c_432242.jpg

Fig.: 14: File request received by the target system

Now let’s simulate the recipient DB system.

Edit file – put the response data in it.

/wp-content/uploads/2014/04/sapbridge_example_44_c_432243.jpg

Fig.: 15: File with response

Move it into the directory books/status. If we were quick enough, the file will be processed by the PI and the response in SOAPUI will look like this:

/wp-content/uploads/2014/04/sapbridge_example_42_c_432245.jpg

Fig.: 16: Response successfully received by SOAP-client

Synchronous-asynchronous bridge successfully built.

All fine, but ..

Just imagine: our customer came and asked us to add one more field to the answer – timestamp.

We don’t have response mapping, the target system also “can’t” provide us response in new format.

What should we do?

/wp-content/uploads/2014/04/bridge17_432249.jpg

Fig.17: Additional development objects in Integration Repository

Also define the mapping, which will convert the response from the system BS_ExtDB in new response format for BS_WebClient.

/wp-content/uploads/2014/04/sapbridge_example_48_c_432250.jpg

Figure 18: Add timestamp to the response

Now configure the routing in Integration Directory:

• create a new “pseudo” File communication channel CC_Dummy_file_receiver and move bridge’s modules DynamicConfigurationBean and NotifyResponseBean there, with all configuration parameters;

• restore the module CallSAPAdapter in communication channel SS_ExtDB_FileSender;

• remove Sender Agreement for SS_ExtDB_FileSender;

• create and configure the Integrated Configuration for response routing from BS_ExtDB to BS_WebClient.

/wp-content/uploads/2014/04/bridge18_432258.jpg

Fig.: 19: Additional configuration objects


Check our bridge as before and we must get the right answer:

/wp-content/uploads/2014/04/sapbridge_example_45_c_432259.jpg

Fig.: 20: Request and response with timestamp.

Ok, the customer is satisfied now, our synchronous-asynchronous bridge works perfect.

Congratulations! 🙂

3.3  Synchronous – asynchronous bridge with modules in Receiver Communication Channel.


There are nothing specific in this version of bridge comparing to the previous. There are the same three modules: two of them – in asynchronous Receiver Communication Channel, transmitting a request to target system; one more – in asynchronous Sender Communication Channel for proceessing the response.


/wp-content/uploads/2014/04/bridge19_432267.jpg

Figure 21 : The logic of the synchronous-asynchronous bridge with modules in the receiver communication channel.


  1. Receiving the synchronous request from external system.
  2. Transfer the message to PI messaging system.
  3. Message processing in PI, transfer it to a synchronous communication channel – to RequestOnewayBean module. The module switch the message processing type from synchronous to asynchronous by changing the message header.
  4. Message goes to the standard adapter module.
  5. Asynchronous call to external system.
  6. Asynchronous response from external system. NotifyResponseBean processing the module instead of standard adapter module.
  7. Transfer request to WaitResponseBean module, correlation check.
  8. The module switch the message processing type from asynchronous to synchronous, transfer the message to messaging sub-system of PI.
  9. Response message is going to synchronous communication channel.
  10. Response to external system.


Modules options are independent from the placement .


Please note: there is another information in SAP help, there are more options for modules when placing in a Receiver Channel. I have not found any meaningful application of these parameters.



RequestOnewayBean parameters:


Parameter

Function

Possible values Default Values
passThrough

Specifies where module must pass the processed message :

– PI message processing subsystem – false

– Next module in the chain – true

true / false false

WaitResponseBean parameters:

Parameter Function Possible values Default Values
timeout Response waiting time in milliseconds 300000

NotifyResponseBean parameters:

Parameter Function Possible Values Default Values
timeout Module waiting time in milliseconds 300000
fault Name of an error message For example, SA_BRIDGE_ERROR
faultNamespace Namespace of an error message:
http://sap.com/xi/XI/System — an error would be count as a system error;
any other namespace – an error would be counted as an application error.

You also should take care about correlation.

3.4 Example : Synchronous – asynchronous bridge with modules in Receiver Communication Channel.

Let’s rebuild our example from 3.2.

/wp-content/uploads/2014/04/bridge20_432411.jpg

Figure 22: Development objects in Integration Repository

At this time we need two synchronous interfaces – SI_WebClient_SaveBook_sync and SI_ExtDB_SaveBook_sync, as well as mapping between them-OM_SaveBook_WebClient_to_ExtDB. We also need asynchronous interface SI_ExtDB_Status (should be based on the same message type as the response in SI_ExtDB_SaveBook_sync) – we will use it for response from an asynchronous system.

Configure routing in Integration Directory as follows:

/wp-content/uploads/2014/04/bridge21_432415.jpg

Fig.: 23: Configuration settings, Integration Directory

Remove all modules from SOAP sender communication channel and move them to file receiver communication channel:

/wp-content/uploads/2014/04/sapbridge_example_53_c_432422.jpg

Fig.: 24: asynchronous file receiver communication channel for request.

/wp-content/uploads/2014/04/sapbridge_example_54_c_432424.jpg

Fig.: 25: asynchronous file receiver communication channel for response.

Reconfigure Integrated Configuration – we have new communication channels for sender and receiver, new interface and mapping; all we have from the previous example – target system only.

To get the file with response, we need only the Sender Agreement connected to the channel CC_ExtDB_File_Sender_WithModule and interface SI_ExtDB_Status.

Start SOAP test tool (SOAPUI in my case) and send test request. You should have a file with request in the file system.

Replace its content with the answer, move it to the folder with responses, wait for PI processing – and you should get following picture:

/wp-content/uploads/2014/04/sapbridge_example_52_c_432431.jpg

Fig.: 26: bridge at work – request and response in SOAPUI.

If your picture looks the same – congratulations! You have just built live sync-async bridge! 🙂

—————————————————————–

See also:

Bridges Construction Site (1/3): Asynchronous – synchronous bridge.

Bridges Construction Site (3/3): SAP PI bridges – “exotic” and recommendations.

—————————————————————–

With best regards,

Alexey Petrov

To report this post you need to login first.

15 Comments

You must be Logged on to comment or reply to a post.

  1. Miten Jain

    Hi Alexey,

       Nice Blog!! I have a query, appreciate if you could help.

    Is there anyway we could set correlation in response async message if Receiver is ABAP Proxy in SAP ECC. I know SAP ECC is synchronous system, but for PPO we went with approach of having ECC as asynchronous system. Our scenario is SOAP sender (sync) – ABAP Proxy(SOAP) as Receiver.

    Thanks,
    Miten

    (0) 
    1. Alexey Pitroff Post author

      Hi Miten,

      try to go this way:

      1) Get message ID in inbound proxy

      http://help.sap.com/saphelp_nw04/helpdata/en/a1/082589fc4246f09793039d5fb01a17/frameset.htm

      or

      http://scn.sap.com/community/pi-and-soa-middleware/blog/2012/02/06/michals-pi-tips-accessing-protocol-classes-for-provider-abap-proxies–xi-message-header

      2) Save it somethere.

      3) Pass it back via client proxy – use additional field in the message structure cause it’s not possible to change message header in client proxy.

      4) Use mapping or Dynamic Configuration Bean to put saved ID into correlationId.

      With best regards,

      Alex

      (0) 
      1. Miten Jain

        Hi Alexey,

           We mapped Message id to Inbound Proxy. In Response mapping we assigned Message id to Adapter attribute and then used Dynamic Configuration Bean in Receiver Agreement of Response.

        It works well. Thank you.

        Regards,

        Miten

        (0) 
    1. Alexey Pitroff Post author

      Hello Erick,

      the main task is to save message GUID and fill CorrelationID parameter in the message header. This can be done in the sender channel too.

      Most adapters (except  JMS) have no standard tools for it.

      So this would be custom solution – with standard module DynamicConfigurationBean, own module,  message mapping or something else.

      In this particular case with file adapter – if I need to use another way – I would think about saving the GUID into the message body.

      With best regards,

      Alex

      (0) 
  2. Erick Carrasco

    Hello Alexey Pitroff.

    Please could you tell me how you would use the message ID in the message body, this point is not clear to me. Please explain to me your example.

    Thanks.

    (0) 
  3. Genica Bocalan

    Hello Alexey,

    I also use Sync Async Bridge and did the trick for the OM. Sync SOAP Sender, Async IDoc Receiver (and some hidden stuffs to do the correlation), but I’m having problems with SAP PI’s processing, cause it forces to do a Response Mapping. Other than that, the Sync SI, when transported, does not overwrite the Async settings because of conflicts (SAP Note: 1580750). Did you manage to solve that issue?

    This is already resolved.

    (0) 
  4. Peter Asigbetse

    Great information Alexey Pitroff.

    I wonder if you anyone can shed some light on how to make this work with

    SOAP ABAP Proxy (Sync) – > PI -> SOAP Webservice (Async)

    I’m using the Get Weather operation at http://www.webservicex.com/globalweather.asmx

    This all works great synchronously.

    I’ve followed the steps above but most blogs all use the FILE adapter as it’s easier to demonstrate but with a SOAP receiver the message get sent but because I’ve used the sync/async pattern no response comes back and it times out.

    I suppose the issue is SOAP async means there’s no response unless there is some sort of callback mechanism that could point it back to a SOAP Sender….but that’s me thinking out loud.

    Can you or anyone give some guidance on how to realise this scenario…get the response form the SOAP service that works synchronously to work asynchronously as part of the sync/async pattern?

    Thanks.

    (0) 
    1. Genica Bocalan

      Hi Peter,

      Good day! Is there another SOAP Web Service that responds with your ‘Request Message’?

      Request Message = ABAP Proxy to SOAP

      Response Message = SOAP to ABAP Proxy

      If yes then you should set an Async – Sync Bridge with your request flow’s receiver channel, set the Sender Interface and Namespace of the message expected to be received there. After that, create an ICo or Classical Configuration using those sender interface and namespace with Virtual Receiver – your virtual receiver system should be the sender system of request flow.

      If not, I don’t think you need to use S/A bridge since its designed to have the response flow. Check with your ABAP team if they need a response and that’s when you’ll need Sync / Async Brdige, if not, change the Service Interface to Async and let your ABAP team regenerate the proxy on their side.

      Best Regards,

      Nica

      (0) 
      1. Peter Asigbetse

        Thanks Genica Bocalan for the prompt response!

        ” Is there another SOAP Web Service that responds with your ‘Request Message’?” – No there isn’t  sadly. I do like your design though. If we did this though you’d never get the response back to the ABAP system would you as it would be an async proxy?


        Check with your ABAP team if they need a response and that’s when you’ll need Sync / Async Bridge” – Yes the response from the service is required as it needs to be acted upon.

        So I’m still stuck with how to get the response back from the service and I’m wondering if the service itself doesn’t fit the scenario. Would a true async web service have some call back mechanism where it would know to direct a response to a PI Sender channel??

        Thanks,

        Peter

        (0) 
        1. Genica Bocalan

          Hello Peter,

          There will be no response if the receiver was an Async SOAP interface (it has HTTP 200 but only a blank soap envelope) but you can do something like this link below.

          The only problem if you’re going to align this reference with your design is, whenever PI splits the message into more than 1 message (which will be like Sender Proxy to Receiver SOAP and Receiver File setup), the message ID changes which will also be the problem of closing the S/A Bridge – because you need to send back the ‘correlation ID’ for PI to identify that response message was the pair of a request message. So better create a file somewhere in PI’s file directory in the mapping part of your build, don’t add another target message in the ICo.

          This is case to case, you can use this for ‘acknowledgement’ only, but if you need the receiver’s response, then they should design another interface for that, or better yet, send the response back to the original SOAP interface.

          This is not the best practice and this poses performance issues, so if you still can change the design, propose Async Proxies. Async Outbound Request and Async Inbound Response. In PI’s request design, Async Outbound Proxy to Async SOAP and Async Inbound Proxy.

          SYNC ASYN Bridge but the receiver is IDOC

          (0) 

Leave a Reply