Not enough JMS queues for Exactly Once? Share them between IFlows!

Introduction

For those coming from a PI background, one of the noticable feature missing on CPI is the capability to implement asynchronous integration with Exactly Once (EO) Quality of Service, and the corresponding ability to reprocess failed messages.

The addition of the JMS adapter (which only works on internal CPI JMS queues) aims to fill this gap by providing the capability to decouple the inbound and outbound message processing of an asynchronous scenario using an internal JMS queue. Once a message is persisted into the JMS queue from the inbound IFlow, it will then be picked up from the JMS queue by the outbound IFlow. Any errors during outbound processing will result in the message being retried again automatically according to the retry setting (interval, exponential backoff). An administrator/developer can also manually retry the message from the JMS queue using Operations view in WebUI. This approach is described in Configure Asynchronous Messaging with Retry Using JMS Adapter.

However, JMS capability is only available on Enterprise Edition Tenants or needs to be purchased separately for Non-Enterprise Edition Tenants. Even with Enterprise Edition, there are resource limits, and by default there are only 30 JMS queues in the tenant. Many organisations with a large footprint on SAP PI will typically have hundreds of asynchronous interfaces, and it becomes a challenge to migrate the interfaces to CPI from both technical and cost perspectives.

Running Out of Queues?

One of the delightful aspect of working with CPI is the flexible modeling environment that is powered by the underlying Apache Camel framework. This combined with the ability to connect separate IFlows using the ProcessDirect adapter really opens up the possibility to come up with endless creative designs to handle integration scenarios – limited only by one’s imagination.

As described in Point 1 of my tips for CPI development, IFlows can be segregated such that common functionalities can be implemented as separate IFlows and invoked using the ProcessDirect adapter. The use cases are limitless, and this blog post covers how it can be utilised from an error handling perspective.

If we were to implement a 1-Interface-1-Queue design, we will run out of the “expensive” JMS queues quickly.

The core idea of this blog post is to implement a common “error handling” IFlow which utilises a single JMS queue for EO/reprocessing functionality, which can be shared with many IFlows via ProcessDirect adapter invocation.

Overview

Following is the outline of the design approach and processing sequence.

1. Integration scenario is split into separate IFlows for inbound message processing (Sender Flow) and outbound message processing (Target Flow).
2. Sender Flow passes message to Target Flow via ProcessDirect adapter.
3. Exception Subprocess is implemented in Target Flow – once error occurs, the message is passed to error handling IFlow (Common JMS Exactly Once Error Handler) via ProcessDirect adapter.
4. In Common JMS Exactly Once Error Handler, initial message is stored into JMS queue.
5. Message is retried from JMS queue and passed back to Target Flow via ProcessDirect adapter.
6. If processing in Target Flow is successful, message status will be Completed and removed from the JMS queue.
7. If any error still occurs, the message remains in the JMS queue and will be retried again at the next interval.

The reason for splitting the scenario to Sender Flow and Target Flow is so that there is an entry point for the error handler IFlow to send the retry messages to.

Note also that the message is only persisted into the JMS queue only if it hits an error during processing. This differs from the approach mentioned in the Introduction section which immediately persists the message into the JMS queue during inbound message processing.

Details of Integration Flows

For the sake of simplicity, the scenario will involve a sender HTTP channel, which triggers a Groovy script logic. Below are the details of each IFlow object.

Sender Flow

The sender IFlow decouples the sender channel from the rest of the processing required for the scenario. Ideally, no other step (or minimal steps) are introduced here so that there is no likelihood of an error occurring at this point. The message is sent to the next IFlow via ProcessDirect receiver channel.

This setup is analogous to a Sender Communication Channel in PI.

Target Flow

The target IFlow contains all the processing logic and steps required for the integration scenario. In this example, it is just a single Groovy script.

The entry point to this IFlow is via a ProcessDirect endpoint – whereby it could receive an initial message from the sender IFlow or a retry message from the error handler IFlow (more details below).

An exception subprocess block is implemented to catch any error that occurs during message processing through the integration process steps. When an error occurs, the exception subprocess appends the following headers into the message, and sends the message to the error handler IFlow’s ProcessDirect endpoint.

• RetryEndpoint – same value as ProcessDirect entry endpoint to this target IFlow
• ErrorMessage – detailed exception message

Message headers can cross IFlows using ProcessDirect adapter, so this allows the subsequent error handler IFlow to know which endpoint to retry the message.

Another important aspect of the design is the following headers need to be allowed in the IFlow configuration. These are headers transferred from the error handler flow when it is retrying a message (more details below).

Common JMS Exactly Once Error Handler

The error handler IFlow is the core of the design. It consists of two integration processes:-

• Top integration process – receives message and stores it into JMS queue
• Bottom integration process – picks up message from JMS queue and sends it to the target IFlow for retry

Following are the details of the key parts of the IFlow:-

Point 1 – ProcessDirect entry endpoint to this error handler. Allows 1/many IFlows to send messages with exception to enable EO retry.

Point 2 – Validates that the RetryEndpoint and ErrorMessage headers are passed by the calling IFlow to ensure that error handling can be processed correctly.

Point 3 – Checks message retry count against the configured max retry limit. If the limit is reached, then message is set to Escalated status and no further processing is done.

Point 4 – Determines if the error is during the initial message processing or during retry. Only initial messages are stored into the JMS queue.

Point 5 – Stores the initial message into the JMS queue.

Point 6 – If the message failed during retry, it is not stored again into the JMS queue to prevent duplicate entries. Instead it is forced to status ‘FAILED’ via a Groovy script, this will cause the existing message that was stored in JMS to be rescheduled for a next retry.

Point 7 – The message will be retried from the JMS queue based on the configured settings in the JMS sender channel.

Point 8 – Header RetryMode is set during retry, so that it can be differentiated at Point 4 above should the message fail again during retry.

Point 9 – Message is retried by sending it back to the target IFlow via the dynamically populated RetryEndpoint header.

Additionally, the following headers are allowed in the IFlow configuration. This allows transfer of details between the target IFlow and the error handler. SAPJMSRetries is an internal header that is automatically populated by the JMS sender channel based on the current retry count of the message.

Note: If the message was successfully processed during a retry, it will change to status ‘COMPLETED’ and will not reenter this error handler IFlow. The message will be removed from the JMS queue upon such successful retry.

Runtime Behavior

To observe the runtime behavior of this design, the Groovy script in Target Flow is designed such that it will raise an exception if the value in the input payload is not an integer. Once the integration scenario is invoked with a non-integer value, the exception will be triggered.

With trace enabled, we see the message flow following the exception route in Target Flow, and the headers ErrorMessage and RetryEndpoint populated.

In the error handler, the initial message is stored in the JMS queue.

It is then picked up from JMS, and sent for retry via the RetryEndpoint header.

If it fails again in Target Flow, it reenters the error handler, but does not get stored again in the JMS queue.

The automatic retry process repeats itself until the message is successful or the retry limit is reached. Note that if ‘Exponential Backoff’ is set, then the retry interval is doubled after each retry (as shown below).

Reference Development Objects

The reference development objects used in this blog post are available in the following GitHub repository.

equalize-cpi-eo-jms-errorhandler

The error handler IFlow (Common JMS Exactly Once Error Handler) is ready to use as-is, and provides the following externalised parameters for further configuration.

Retry related settings for JMS queue

Expiration and JMS queue name settings

Maximum retry setting

Conclusion

With the design described in this blog post, we can significantly improve the error handling aspects of asynchronous interfaces in CPI. This design is by no means “set in stone”, but meant to showcase how flexible Camel-based modeling and ProcessDirect adapter come together to allow us to design creative solutions to integration requirements.

The approach can even be further enhanced in the following manner:-

• trigger email notification if maximum retry limit have been reached.
• instead of using a single error handler (single JMS queue) for all interfaces, deploy mulitple error handler IFlows for different categories of interfaces (based on priority, volume, business category, etc).