Cloud Integration – JMS Resource and Size Limits

This blog describes the resource and size limits for JMS resources used in the JMS, XI and AS2 adapter, that can be used in Cloud Integration Enterprise Edition and Integration Suite Standard and Premium. It describes which limits apply and how to monitor them. It also gives some guidance on how to cleanup the resources.

JMS Resource and Size Limits

The connected JMS messaging instance, that is used in asynchronous messaging scenarios using with the JMS, XI or AS2 Adapter has limited resources.

There are also technical restrictions on the size of the headers and exchange properties that can be stored in the JMS queue.

Resource Limits for SAP Cloud Integration Enterprise Edition and in SAP Integration Suite

In the Cloud Integration Enterprise Edition and in the Integration Suite, a dedicated set of resources is provisioned per default in the JMS instance, which can be introduced via self-service. Keep this in mind when configuring and running your scenarios.

Default provisioned resources:

• Number of Queues in Cloud Integration: 30
• JMS Resources for Monitoring
• Total Queue Capacity: 9,3GB
• Maximum Capacity for one Queue: 95% of the Total Queue Capacity
• Transactions: 150
• Consumers: 150
• Providers: 150
• Maximum Messages in one Transaction: 256
• Message Volume: 150GB/month

It is possible to increase the JMS resources via self-service for up to 100 queues as described in detail in the blog Activating and Managing Enterprise Messaging Capabilities.

NOTE: If you have a use case of requiring more than 100 queues, you need to create an incident following the KBA 2589823

It is possible to assign additional Enterprise Messaging units containing one queue and a dedicated set of JMS resources.

5 Enterprise Messaging Units consist of:

• Number of JMS Queues in Cloud Integration: 1
• Queue Capacity: 300MB
• Transactions: 5
• Consumers: 5
• Providers: 5
• Message Volume: 5GB/month

Sample resources for 100 message queues:

• Number of JMS Queues in Cloud Integration: 100
• Queue Capacity: 30GB
• Transactions: 500
• Consumers: 500
• Providers: 500
• Message Volume: 500GB/month

Limits not related to the numbers of Enterprise Messaging Units:

• Maximum Capacity for one Queue: 95% of the Total Queue Capacity
• Maximum Messages in one Transaction: 256

JMS Resources

The JMS Resources are shown in the Queue Monitor in the Manage Storages section in the Operations view:

If the JMS Resources get critical the message appears as a yellow warning message. If the JMS resources are exhausted, it appears as a red error message.

If the resources are exhausted, messages can no longer be processed, so you need to keep an eye on this value. Make sure that there are no messages stuck in queues!

Using the Details link you get the detailed overview of the JMS Resources:

How to activate alerting for the JMS Resources is described in the blog Automated Notification for Critical or Exhausted JMS Resources.

Number of Queues

JMS queues are used by the JMS, AS2 and XI adapter. When the first integration flow that uses a JMS queue is deployed, the queue is created in the messaging broker. The queue can then be used for sending messages into the queue and for consuming messages from the queue.

The status for Number of Queues gets Critical if only one queue is left. In the above case it would get the critical status when 28 queues are created and only one is left.

The status for Number of Queues is never Exhausted because there is no impact in runtime. If you would deploy a new integration flow using a new queue you would get an error during deployment of the integration flow.

Queue Deletion

If you no longer need a specific queue, (for example if the integration flow has been undeployed), you have to delete the queue manually in the Queue Monitor.

If the queue limit has been reached, you can check for unused queues and delete them. For more information about checks for unused queues, see the blog ‘Checks in Queue Monitor’.

Total Queue Capacity

The total queue capacity used by all the scenarios is shown in the JMS Resources.

If the overall capacity used reaches 80% of the total capacity available, the status gets Critical. If 95% of the available capacity is used, the status gets Exhausted/Error.

If 100% of the total queue capacity is reached, messages can no longer be processed in the queues, so you need to keep an eye on this value or setup an automated notification as suggested above. Make sure that consumers are registered for the queues and that no messages are stuck in queues!

Queue Status

In addition to the overall queue capacity there is also a limit per queue. One queue might currently use 95% of the total queue capacity. With the 8-December-2019 release you have the option to configure this size for each of your JMS queues separately. This feature allows you to prevent that one overloaded queue affects other scenarios.

If at least one queue has a warning status, the Queue Status goes to Critical shown in orange color, if at least one queue has an error status, the Queue Status goes to Error and is shown in red.

The master view of the queue monitor shows the status for the individual queues. The Status column shows a warning or error icon if the size of this queue reaches a critical limit. The warning icon is shown if more than 80% of the queue size is reached, if 95% or more of the configured queue size is reached an error icon is shown.

To get the detailed status for the queue, click on the Status action from the menu:

You can see that the selected queue has three sub-queues required for the processing of the messages:

• Processing Queue: Here, the main processing takes place. Messages are stored there using the JMS receiver and are consumed from it using the JMS sender.
• Error Queue: Here, messages in error are parked until the next configured retry. From this queue messages are not consumed; the messages are put back to the processing queue when the configured retry interval is reached.
• Chunking Queue: Messages larger than 5MB are split into one 5MB parent message and multiple chunks, because the message broker does not support messages over a certain message size. The single chunk messages have a size limit of a maximum of 5MB. The parent message is stored in the processing queue while the multiple chunks are stored in the chunking queue.

For example, a 46MB message would be split into one 5MB parent message stored in the processing queue, and eight 5MB and one 1MB chunk messages stored in the chunking queue.

In case such a large message runs into an error, the parent message is moved to the error queue, but the chunk messages are kept in the chunking queue. The retry is triggered once the parent message is moved back to the processing queue after the configured retry interval.

It is important to understand the difference between the sub-queues when you configure the sizes as the sizes can be configured on each of the sub-queues. The size can be configured using the Configure Size option in the Queue Status view or the Configure Size action in the menu for the queue:

In the Configure Queues Sizes view you define the size for each sub-queue in MB. The default for each queue is 95% of the total queue capacity on the broker. The size can be configured to any value between 100MB and 95% of the total queue capacity.

Note, that you can also configure a size that is lower than the current queue size. In this case no further messages can be sent to the queue until enough messages are consumed from this queue and free space is available.

Active/Stopped Queue Status

In the master view a queue is also shown with a red error sign if it is in Stopped status:

A stopped queue can be started with the Start action in this Queue Status View.

If there are problems in an integration scenario and you want to avoid that the JMS resources are eaten up by this scenario you are able to stop a queue. A stopped queue is not processing any messages anymore, it is neither possible to store messages anymore in this queue, not consume from this queue. This can be done as administrative action via the action Stop in the Queue Status View for a queue:

Transactions, Consumers and Providers

To process a message during runtime always a consumer or provider and an open transaction are required. If a message is stored to JMS queue (JMS Receiver channel) a provider is required, for polling a message from JMS queue (JMS sender channel) a consumer is required.

If the JMS Resources get Critical you need to optimize the usage of transactions in consumers and providers or increase the message queue quota. If more than 100 message queues or equivalent JMS resources are required follow note 2589823.

Think about the following technical details:

Consumers

Consumers are created in the JMS broker from JMS sender, AS2 sender and XI sender and receiver adapters to consume messages from a JMS queue. There is a minimum number of consumers that are always created for the deployed integration flows having JMS queues, independent of the message throughput, and a maximum number that is started when many messages are stored in the JMS queues and need to be consumed.

Minimum number of consumers:

For each JMS queue used in a JMS, AS2  or XI sender or receiver channel a minimum of one consumer is created for each runtime node started in the Cloud Integration cluster.

In short, the minimum number of consumers for a tenant can be calculated like this:

C_min = number of worker nodes * number of JMS queues

Maximum number of consumers:

If lots of messages are to be consumed from the JMS queue, the initially created consumer is dynamically increased up to the configured Number of Concurrent Processes in the JMS or AS2 channel to provide a higher message throughput. For each JMS queue used in a JMS or AS2 sender channel, a maximum of consumers is created as concurrent processes are configured in the JMS or AS2 sender channel in parameter Number of Concurrent Processes. If several runtime nodes are started in the Cloud Integration cluster those many consumers are created from each runtime node.

If the XI sender or receiver adapter is used with a JMS queue as temporary storage, only one consumer is started.

In short, the maximum number of consumers for a tenant can be calculated like this:

C_max = number of runtime nodes * total number of concurrent processes in all JMS queues

Note that the numbers of runtime nodes is shown in the JMS Resources view after the 25-November-2018 update at the very end of the screen. Before the November update this was not explicitly shown in the WebUI. The only option you had then, was to count the numbers of system log files (ljs_trace_*) written in parallel in your tenant. In the System Log Files monitor you could check how many ljs_trace* files are written at the same time. It’s one for each runtime node.

Note, that additional consumers are created for monitoring (when the Queue Monitor is used) and in case large messages are processed.

To reduce the parallel consumers, you can either reduce the number of concurrent processes in the JMS sender channels or reduce the number of runtime nodes started in the cluster.

Providers

Providers are created by the JMS receiver, AS2 sender and XI sender and receiver adapter in the JMS broker to store messages in a JMS queue. As many providers are created as messages are send to the JMS queue in parallel. The number of providers created is dynamically increased as long as enough providers and transactions are available.

In short, the number of providers for a tenant cannot be calculated, but depend on the sender system:

P = number of parallel sender calls

Note, that additional providers are created in case large messages are processed.

To reduce the parallel providers, you need to reduce the parallelism of the inbound processing by reducing the number of parallel inbound calls from sender systems sending messages to scenarios using JMS queues.

Transactions

To consistently process messages, JMS transactions are required in the JMS broker to be able to consistently roll back the processing in case of errors. As many transactions are created in the JMS broker as consumers and providers are under processing in parallel. This means, that for every consumer and every provider a transaction is started.

In short, the minimum number of transactions for a tenant can be calculated like this:

T_min = C_min + P

And the maximum number of transactions for a tenant can be calculated like this:

T_max = C_max + P

There are 150 open transactions available in the JMS broker, they can be used for a consumer or for a provider. The transactions are distributed dynamically to providers and consumers. This means, if lots of messages are consumed in parallel in consumers (polling messages form JMS queues via JMS, AS2 sender or XI sender and receiver channels) most of the transactions will be used for those and the number of available transactions to store message to JMS queues is lower.

If the limit for transactions is reached (if you get a runtime error) you need to reduce the parallelism for the consumers and/or providers (see above).

If multiple JMS resources, like JMS, XI or AS2 Sender Adapters and/or one or more JMS Receiver Adapters are used in one integration flow you can optimizes the numbers of used transactions in the JMS instance using a JMS transaction handler because then only one transaction is opened for the whole processing. More details about this option you can find in blog ‘How to configure transaction handling in integration flow/‘.

Numbers of Messages in one Transaction

There is a maximum of 256 messages allowed in one transaction. This is important to know for split scenarios, where multiple split messages are executed in the same transaction. If there are more splits, a runtime error is thrown when the messages are committed in the JMS broker. The following scenarios split messages:

• Large Messages: If a message is bigger than 5MB, it is internally split into chunks before storing it in the JMS queue. This means if a 50MB message is received it is split in 10 chunks and so 10 messages are contained in one transaction. This means the message to store in a JMS queue must not exceed 1280MB (256*5MB).
• Splitter scenarios: If the original message is split into multiple split parts by the Splitter flow step you need to make sure not more than 256 split parts are created and stored in a JMS queue using the JMS receiver adapter. One workaround option for such a scenario would be to first split the message into bigger chunks using a Grouping in the Splitter and store the chunks in a first JMS queue (kind of intermediate queue). In a second process those bigger chunks could be split into single entities using a JMS sender adapter to read the chunks and store the single splits in the final JMS queue using JMS receiver adapter.

Size Limits for Messages, Headers, Properties and Attachments

The following size limits apply when saving messages to JMS queues.

• There maximum message size (including attachments) that can be stored in a JMS queue is 1280MB. See in last chapter for more details).
• Headers and exchange properties defined in the integration flow before the message is saved to queue must not exceed 4MB for headers and properties together.

Important: As mentioned, the size limits for payload and attachments are quite high, but you need to keep in mind that processing the message in the CPI runtime may limit the possible size of payload and attachments for your scenario. You as scenario developer have to test and restrict the limits your scenario can handle!