Skip to Content

This blog describes options for configuring asynchronous one-way message processing using the new JMS adapter, which is now available for customers. It describes the setup, configuration, size limits and monitoring of your scenario. The JMS adapter is available for SAP Cloud Platform Integration customers with an Enterprise Edition license since 24-June-2017 release (2.29*).

Asynchronous Messaging with Retry Using the JMS Adapter

In many cases integration scenarios have to be decoupled asynchronously between sender and receiver message processing to ensure that a retry is done from the integration system rather than the sender system. This can be achieved, for example, by using JMS queues to temporarily store the messages in the cloud integration system if the receiver system cannot be reached, and retry them from there. Follow the steps described below to activate the message broker and to setup the scenario. Use the monitoring tools described to check the processing and view the stored messages in the queue monitor.

Prerequisite: Broker Provisioning

To be able to use JMS queues for storing messages you first need to activate the message broker for your cloud integration tenant. If you have an Enterprise Edition license you can do this for your tenant in Cloud Integration Provisioning via Provision Broker. You can see then the active message broker and its details. You can now start using the JMS adapter to configure your scenario.

This is described in detail in blog ‘How to activate Message Broker’.

Setup Scenario With Asynchronous Decoupling

To configure the decoupling of inbound and outbound message processing you need to configure two processes:, one process to receive the inbound message and store it in the JMS queue and a second process to trigger the message from the JMS queue to the receiver backend. The blog describes the configuration using two separate integration flows.

Configure the Integration Flow Receiving the Message

The first integration flow will be configured to receive the message via any inbound adapter. In this sample setup we use the HTTP adapter.

Configure the JMS Receiver Channel

Create the integration flow with the inbound channel required by your scenario, and use the JMS adapter as the outbound adapter. You only have to configure the Queue Name for the JMS queue that will be used to store the messages. It is Important to set an Expiration Period that suits your scenario. The default value is 90 days’, which means that after 90 days any messages, that were not delivered within this period will be deleted from the queue.

With the 10-December 2017 release, you will have the option to configure, if exchange properties are forwarded to the JMS queue. Before, the properties were always forwarded along with the message, which caused severe issues in some scenarios. The default value of the option will be, that properties are not forwarded, because this could lead to runtime issues if the properties are too large. In detail the size restrictions are described in the Blog ‘JMS Resource ans Size Limits’.

This option is available in adapter version 1.1.

Configure Transaction Handling

You need to configure the correct transaction manager in the integration process for transactional end-to-end processing. Our process does not contain any JDBC resources (data store, variables, aggregator), and only one direct JMS Receiver without splitter or sequential multicast; so we don’t need a specific transaction handler. Select the integration process and switch to the Processing tab. In Transaction Handling drop down select Not Required.

More details about the different transaction handling options and existing limitations are described in the blog ‘How to Configure Transaction Handling in Integration Flow’.

Deploy the Integration Flow

Now you can deploy the integration flow. During deployment, the specified queue will be created automatically in the message broker. You can check that the queue has been created in the Queue Monitor available in the operations view in the Manage Storages section. The monitor is described in more detail in the Monitoring section below.

If a message broker has not yet been provisioned, the deployment will end with an error. In Manage Integration Content the error details for your integration flow tell you that no messaging host is available.

Configure the Integration Flow Doing the Retry

To consume the messages from the JMS queue, you configure a second integration flow with a JMS sender channel and the outbound adapter needed for your scenario. In this sample configuration we use the HTTP adapter.

Configure the JMS Sender Channel

Create the integration flow with the outbound channel required by your scenario, and use the JMS adapter as the inbound adapter. You only have to configure the Queue Name for the JMS queue that you want to poll the messages from. Use the same queue name used in the receiving integration flow.

Number of Concurrent Processes is set to 1 per default and should only be increased if more parallel processing is needed. Because the JMS resources are limited, you need to keep the number of parallel processes as low as possible. Especially if large messages are processed in the scenario, increasing the parallelism may lead to out of memory problems.

In the JMS sender channel you also have to configure the retry handling. Set the Retry Interval as required for your scenario (, the default value is 1 minute). We also recommend using Exponential Backoff, which means that the retry interval is doubled after each unsuccessful retry. By using this setting you can avoid calling the receiver backend every minute if it is unavailable for a longer time period. The Maximum Retry Interval defines the maximum time between two retries if exponential backoff is used. Make sure you do not set this value too high as this can result in very long intervals between retries.

With 22-July-2017 release (2.30*) the option to configure a dead letter queue is introduced. This is described in detail in blog How to configure Dead Letter Handling in JMS Adapter.

Configure Transaction Handling

If you are using JMS Sender, it is not necessary to set the transaction handling to Required for JMS, because the retry handling works independently of the selected transaction handler. You should select the transaction manager as required by the scenario that you have configured in the integration flow.

In the simple example shown, no JDBC or JMS resources are used, that need a transaction handler, so we select Not Required.

There are some limitations on configuring transaction handling configuration and on which flow steps can be used in which combinations. This is described in detail in the blog ‘How to configure Transaction Handling in Integration Flow‘.

Optional: Explicit Retry Configuration

In the JMS sender channel, you can configure the time intervals between retries, but you cannot configure that processing will end after a specific number of retries. If required, you can configure this in an exception subprocess that calls a local process for retry handling.

To configure this, add a Local Integration Process to the integration flow. In this process you configure the specific retry handling. In this sample, we check the number of retries executed and after 5 unsuccessful retries, we end the processing and trigger a mail to alert someone to the problem.

In the local integration process configure a Router after the start event, add an Error End Event and configure the router as shown in the picture. The header SAPJMSRetries set by the JMS adapter is used to decide if the message processing continues or if it ends and a mail is triggered. If message processing is to continue, an error is raised by the process so that the message stays in the JMS queue and goes into retry status.

We set the transaction handling to From Calling Process in the local process, and as there are neither JMS nor JDBC transacted resources configured in the local process, we take over the Not Required setting from main process.

Furthermore, you need to add an Exception Subprocess in the main process, and add a Local Process Call, where you select the configured local process, within the exception subprocess. Add a Receiver for the mail to be sent. Connect the Message End Event of the exception subprocess to the receiver using a Mail adapter receiver channel. In this channel, you configure the mail address the mail is to be sent to and the details to be sent.

With this configuration the message will be removed from the JMS queue after 5 unsuccessful retries, so that no further retries are executed. A mail is triggered to notify the appropriate person to take manual action.

If you use the explicit retry configuration, you are free to configure retries as required for your scenario. For example, you can notify someone after 3 retries but not stop the processing until 10 retries have been made, to allow enough time for manual actions. Or, you can send the message after 3 retries via an alternative receiver channel.

Deploy the Integration Flow

You can now deploy the integration flow. This completes the scenario configuration in SAP Cloud Platform Integration and you can start processing messages. Test the configuration by sending a test message via the configured inbound adapter and check that it is received by the receiver backend.

JMS Resource and Size Limits

The connected JMS messaging instance, that is used in the asynchronous messaging scenario has limited resources. The Enterprise License sets a limit on the queues, storage and connections that you can use in the messaging instance. The limits are described in more detail in the blog ‘JMS Resource and Size Limits‘.

Monitoring

To check the processing in detail, you can use the monitoring tools provided by Cloud Integration.

Integration Flow Monitoring

In the operations view, in the section Manage Integration Content, you can find the deployed integration flows and their status. Check that the flows have the status Started.

Message Monitoring

Message processing can be checked in detail in the section Monitor Message Processing. Select the message you sent and check the processing status. There should be two message processing logs, one for the inbound messaging that saves the message to the JMS queue, and one for processing the message from the queue and sending it to the receiver system.

In the event of a failure when sending to the receiver system the outbound message will go into status Retry. The message will then be retried as configured in the JMS sender channel. The detailed error information and the retries are available under the link Message Processing Log.

Queue Monitor

If there are problems during processing and your message does not arrive at the receiver backend, the message status is Retry and the messages can be checked in the Queue Monitor in the section Manage Storages in the Operations view. This monitor shows all queues existing in your tenant, with their stored messages. Select your queue to check the messages contained in the queue.

You can see the number of retries that have already been executed and the time of the next retry. You can also trigger an immediate retry of the message or delete specific messages if you no longer want them to be processed.

If a message broker has not yet been provisioned, the Queue Monitor tile is not shown.

Queue Deletion

If you do no longer need a specific queue, (for example if you un-deploy the integration flow), you have to delete the queue manually in the Queue Monitor. The queue cannot be deleted automatically because this could cause inconsistencies, as there may still be messages in the queue that would still have to be processed.

To find unused queues in the monitor you may use the checks described in the blog ‘Checks in JMS Message Queue Monitor’.

To report this post you need to login first.

5 Comments

You must be Logged on to comment or reply to a post.

  1. Vijayashankar Konam

    Thanks  for the detailed blog Mandy . This had been a long pending feature on SCI. However, hope SAP implemented the EO and EOIO QoS out of the box rather than customers to develop their own scenarios.

    (0) 
  2. Beverely Parks

    Thank you for this blog.  It is very helpful.  I am just starting to attempt to use JMS queues.  How do you recommend setting up the JMS queues?  One per message type or one per Sender?  We receive multiple transaction types from a Sender with an end point for each message type.  Would you recommend creating a queue for each message type/end point or for the Sender?  If at the Sender, how would the actual processing IFLOW look?

    These messages are being sent to our On-Premise ECC system.  I’m looking into JMS queues as a means of reprocessing transactions that we receive while our On-Premise ECC system is down for maintenance.  

    (0) 
    1. Mandy Krimmel Post author

       

      Hello Beverely,

      in general both is possible, you could route the messages based on some routing condition (based on payload) into different queues, or put all in one queue directly after receiving and do routing based on message type later. I would recommend second, because so you save the messages directly after receiving and have the retry point there. Also, you only have a limited number of queues available, see https://blogs.sap.com/2017/10/04/cloud-integration-jms-resource-and-size-limits-in-cpi-enterprise-edition/ .

      BR,

      Mandy

      (1) 
  3. Beverely Parks

    Thank you for your response.  Please understand that I am new to Cloud Integration with no PI experience.  I am stuck on two points.

    #1 – We receive multiple transaction types from a 3rd party.  Let’s say Orders, Shipments, and Receipts.  We currently have one IFLOW for each transaction.  Upon receiving each, we assign the SAP application ID, save a copy of the payload, perform mapping and send the transaction to our On-Premise ECC system.  So IFLOW 1 sends to ECC address#1, IFLOW 2 sends to ECC address#2, etc.  Each IFLOW also contains an Exception sub-process to send an email notification in case of failure.   Is it best to send the data to the JMS queue before any of the current IFLOW processes, or to send the data to the JMS queue rather than directly to ECC On-Premise?

    #2 If I use just one JMS Queue for all 3 transactions, how do I read the transaction name (ns1) to determine the routing?

     

    Thank you for your time and any assistance that you are able to provide.

    (0) 
    1. Mandy Krimmel Post author

       

      Hello Beverely,

      as already said, both is possible:

      1. use the existing flows and just do async decoupling in each of them
      2. create one receiving flow with a JMS receiver channel and put all messages in one JMS queue and have a second flow polling the messages out, routing the messages based on a routing condition (e.g. via XPath in router flow step – more info can be found in online help) to the different receivers

      Advantages for first option:
      – you can keep the existing flows and just introduce decoupling in all the flows
      – flows/scenarios are independent
      — flows can be updated independently
      — higher throughput possible

      Advantages for second option:
      – one central processing flow, may be easier to maintain in futur
      – only one queue, may be an aspect if you need several scenarios with JMS, as only 28 queues are allowed in enterprise edition

      Best regards,
      Mandy

      (0) 

Leave a Reply