Skip to Content
Technical Articles

Cloud Integration – JMS Resource and Size Limits

This blog describes the resource and size limits for JMS resources used in the JMS, XI and AS2 adapter, that can be used in Cloud Platform Integration Enterprise Edition. It describes which limits apply and how to monitor them. It also gives some guidance on how to cleanup the resources.

JMS Resource and Size Limits

The connected JMS messaging instance, that is used in asynchronous messaging scenarios using with the JMS, XI or AS2 Adapter has limited resources. The Enterprise License sets a limit on the queues, storage and connections that you can use in the messaging instance.

There are also technical restrictions on the size of the headers and exchange properties that can be stored in the JMS queue.

Resource Limits for SAP Cloud Platform Integration Enterprise Edition

In the CPI Enterprise Edition, a dedicated set of resources is available in the JMS instance. Keep this in mind when configuring and running your scenarios.

  • Maximum Number of Queues in Cloud Integration: 30
  • JMS Resources for Monitoring
  • Total Queue Capacity: 9,3GB
  • Maximum Capacity for one Queue: 4GB (after 2019-Feb-17: 95% of the Total Queue Capacity)
  • Transactions: 150
  • Consumers: 150
  • Providers: 150
  • Maximum Messages in one Transaction: 256
  • Message Volume: 150GB/month

Starting January 2019 it is possible to increase the JMS resources in the CPI Enterprise Edition tenant. This is described in detail in the blog Activating and Managing Enterprise Messaging Capabilities.

It is possible to assign additional Enterprise Messaging units containing one queue and a dedicated set of JMS resources.

5 Enterprise Messaging Units consist of:

  • Number of JMS Queues in Cloud Integration: 1
  • Queue Capacity: 300MB
  • Transactions: 5
  • Consumers: 5
  • Providers: 5
  • Message Volume: 5GB/month

Resource Limits for other SAP Cloud Platform Integration Editions

Starting January 2019 it is possible to assign dedicated JMS resources also to Cloud Integration tenants not having the CPI Enterprise Edition. This is described in detail in the blog Activating and Managing Enterprise Messaging Capabilities.

It is possible to assign units containing one queue and a dedicated set of JMS resources.

5 Enterprise Messaging Units consist of:

  • Number of JMS Queues in Cloud Integration: 1
  • Queue Capacity: 300MB
  • Transactions: 5
  • Consumers: 5
  • Providers: 5
  • Message Volume: 5GB/month

Limits not related to the numbers of Enterprise Messaging Units:

  • Maximum Capacity for one Queue: 4GB (after 2019-Feb-17: 95% of the Total Queue Capacity)
  • Maximum Messages in one Transaction: 256

NOTE: You need to buy a minimum quantity of 10 units of Enterprise Messaging SKU to get one JMS queue activated in Cloud Integration because you require 5 additional units for the JMS monitoring.

Examples:

  • for one JMS queue in Cloud Integration you need 10 units (5 for the queue, 5 for monitoring)
  • for two JMS queues in Cloud Integration you need 15 units (10 for the 2 queues, 5 for monitoring)
  • for three JMS queues in Cloud Integration you need 20 units (15 for the 3 queues, 5 for monitoring)

NOTE: You always have to buy in multiplication of 5.

JMS Resources

The JMS Resources are shown in the Queue Monitor in the Manage Storages section in the Operations view:

If the JMS Resources get critical the message appears as a yellow warning message. If the JMS resources are exhausted, it appears as a red error message.

If the resources are exhausted, messages can no longer be processed, so you need to keep an eye on this value. Make sure that there are no messages stuck in queues!

Using the Details link you get the detailed overview of the JMS Resources:

How to activate alerting for the JMS Resources is described in the blog Automated Notification for Critical or Exhausted JMS Resources.

Number of Queues

JMS queues are used by the JMS, AS2 and XI adapter. When the first integration flow that uses a JMS queue is deployed, the queue is created in the messaging broker. The queue can then be used for sending messages into the queue and for consuming messages from the queue.

The status for Number of Queues gets Critical if only one queue is left. In the above case it would get the critical status when 32 queues are created and only one is left.

The status for Number of Queues is never Exhausted because there is no impact in runtime. If you would deploy a new integration flow using a new queue you would get an error during deployment of the integration flow.

Queue Deletion

If you no longer need a specific queue, (for example if the integration flow has been undeployed), you have to delete the queue manually in the Queue Monitor. Unused queues, that do not contain messages and are not referred by an Integration Flow on runtime will be deleted automatically.

If the queue limit has been reached, you can check for unused queues and delete them. For more information about checks for unused queues, see the blog ‘Checks in Queue Monitor’.

Queue Capacity

The total queue capacity used by all the scenarios is shown in the JMS Resources.

If the overall capacity used reaches 80% of the total capacity available, the status gets Critical. If 95% of the available capacity is used, the status gets Exhausted/Error.

If the limit is reached, messages can no longer be processed, so you need to keep an eye on this value. Make sure that there are no messages stuck in queues!

Limit per Queue

In addition to the overall queue capacity there is also a limit per queue. This limit is currently 4GB. After the 2019-Feb-17 update the value is set to 95% of the Total Queue Capacity. But note, for existing integration flows a redeployment of the integration flow is required to increase this value for the queue.

The master view of the monitor shows how this capacity is distributed between the queues. The Size column displays the size of all messages in this queue.

 

Transactions, Consumers and Providers

To process a message during runtime always a consumer or provider and an open transaction are required. If a message is stored to JMS queue (JMS Receiver channel) a provider is required, for polling a message from JMS queue (JMS sender channel) a consumer is required.

If the JMS Resources get Critical you need to optimize the usage of transactions in consumers and providers. Think about the following technical details:

Consumers

Consumers are created in the JMS broker from JMS sender, AS2 sender and XI sender and receiver adapters to consume messages from a JMS queue. There is a minimum number of consumers that are always created for the deployed integration flows having JMS queues, independent of the message throughput, and a maximum number that is started when many messages are stored in the JMS queues and need to be consumed.

Minimum number of consumers:

For each JMS queue used in a JMS, AS2  or XI sender or receiver channel a minimum of one consumer is created for each runtime node started in the Cloud Integration cluster.

In short, the minimum number of consumers for a tenant can be calculated like this:

Cmin = number of worker nodes * number of JMS queues

Maximum number of consumers:

If lots of messages are to be consumed from the JMS queue, the initially created consumer is dynamically increased up to the configured Number of Concurrent Processes in the JMS or AS2 channel to provide a higher message throughput. For each JMS queue used in a JMS or AS2 sender channel, a maximum of consumers is created as concurrent processes are configured in the JMS or AS2 sender channel in parameter Number of Concurrent Processes. If several runtime nodes are started in the Cloud Integration cluster those many consumers are created from each runtime node.

If the XI sender or receiver adapter is used with a JMS queue as temporary storage, only one consumer is started.

In short, the maximum number of consumers for a tenant can be calculated like this:

Cmax = number of runtime nodes * total number of concurrent processes in all JMS queues

Note that the numbers of runtime nodes is shown in the JMS Resources view after the 25-November-2018 update at the very end of the screen. Before the November update this was not explicitly shown in the WebUI. The only option you had then, was to count the numbers of system log files (ljs_trace_*) written in parallel in your tenant. In the System Log Files monitor you could check how many ljs_trace* files are written at the same time. It’s one for each runtime node.

Note, that additional consumers are created for monitoring (when the Queue Monitor is used) and in case large messages are processed.

To reduce the parallel consumers, you can either reduce the number of concurrent processes in the JMS sender channels or reduce the number of runtime nodes started in the cluster.

Providers

Providers are created by the JMS receiver, AS2 sender and XI sender and receiver adapter in the JMS broker to store messages in a JMS queue. As many providers are created as messages are send to the JMS queue in parallel. The number of providers created is dynamically increased as long as enough providers and transactions are available.

In short, the number of providers for a tenant cannot be calculated, but depend on the sender system:

P = number of parallel sender calls

Note, that additional providers are created in case large messages are processed.

To reduce the parallel providers, you need to reduce the parallelism of the inbound processing by reducing the number of parallel inbound calls from sender systems sending messages to scenarios using JMS queues.

Transactions

To consistently process messages, JMS transactions are required in the JMS broker to be able to consistently roll back the processing in case of errors. As many transactions are created in the JMS broker as consumers and providers are under processing in parallel. This means, that for every consumer and every provider a transaction is started.

In short, the minimum number of transactions for a tenant can be calculated like this:

Tmin = Cmin + P

And the maximum number of transactions for a tenant can be calculated like this:

Tmax = Cmax + P

There are 150 open transactions available in the JMS broker, they can be used for a consumer or for a provider. The transactions are distributed dynamically to providers and consumers. This means, if lots of messages are consumed in parallel in consumers (polling messages form JMS queues via JMS, AS2 sender or XI sender and receiver channels) most of the transactions will be used for those and the number of available transactions to store message to JMS queues is lower.

If the limit for transactions is reached (if you get a runtime error) you need to reduce the parallelism for the consumers and/or providers (see above).

If multiple JMS resources, like JMS, XI or AS2 Sender Adapters and/or one or more JMS Receiver Adapters are used in one integration flow you can optimizes the numbers of used transactions in the JMS instance using a JMS transaction handler because then only one transaction is opened for the whole processing. More details about this option you can find in blog ‘How to configure transaction handling in integration flow/‘.

Numbers of Messages in one Transaction

There is a maximum of 256 messages allowed in one transaction. This is important to know for split scenarios, where multiple split messages are executed in the same transaction. If there are more splits, a runtime error is thrown when the messages are committed in the JMS broker. The following scenarios split messages:

  • Large Messages: If a message is bigger than 5MB, it is internally split into chunks before storing it in the JMS queue. This means if a 50MB message is received it is split in 10 chunks and so 10 messages are contained in one transaction. This means the message to store in a JMS queue must not exceed 1280MB (256*5MB).
  • Splitter scenarios: If the original message is split into multiple split parts by the Splitter flow step you need to make sure not more than 256 split parts are created and stored in a JMS queue using the JMS receiver adapter. One workaround option for such a scenario would be to first split the message into bigger chunks using a Grouping in the Splitter and store the chunks in a first JMS queue (kind of intermediate queue). In a second process those bigger chunks could be split into single entities using a JMS sender adapter to read the chunks and store the single splits in the final JMS queue using JMS receiver adapter.

Size Limits for Messages, Headers, Properties and Attachments

The following size limits apply when saving messages to JMS queues.

  • There maximum message size (including attachments) that can be stored in a JMS queue is 1280MB. See in last chapter for more details).
  • Headers and exchange properties defined in the integration flow before the message is saved to queue must not exceed 4MB for headers and properties together.

Important: As mentioned, the size limits for payload and attachments are quite high, but you need to keep in mind that processing the message in the CPI runtime may limit the possible size of payload and attachments for your scenario. You as scenario developer have to test and restrict the limits your scenario can handle!

7 Comments
You must be Logged on to comment or reply to a post.
  • Hello Mandy,

     

    thanks for the Blog. We just got the feedback from support that there is also a limit per queue (4GB). Is this still accurate or not?

    Thanks and kind regards

     

    • Hello Richard,

      yes, there also is a limit of 4GB on each queue. This value is not (yet) configurable.

      I added this now also to the blog.

      Best regards,

      Mandy

  • Hello Mandy,

    It seems that with the new capability to enable Enterprise Messaging within SAP CPI we  have now two mechanisms to enable asynchronous queuing over JMS in SAP CPI:

    • JMS Message Broker (Just consumable within CPI IFlows).
    • SAP Enterprise Messaging (Neo Environment. Just consumable within CPI IFlows)

    For new SAP CPI customers with Enterprise Edition license, what is the SAP recommended option? What is the way forward? What are the Pros and Cons of each option?

    Many thanks for your time.

    Kind regards,

    C.

    •  

      Hello,

       

      this is a misunderstaning. JMS Message broker and SAP Enterprise Messaging use exactly the same functionality underneath, this are no different options. Both us the same JMS message broker underneath.

      CPI with Enterprise Editions uses Enterprise Messaging and if you want to increase the JMS limits you have to buy additional Enterprise Messaging units.

      For non-enterprise licenses you can buy Enterprise Messaging separately.

      BR,

      Mandy

  • Hi Mandy,

    How to delete these queues where messages are in ‘Retry’ Status ?

    I am having All authorizations(Administrator as well) but getting message like “Not Authorized to perform the operation”.

    Sender is AS2 and we have Enterprise Edition.

    let me know how I can delete these queues.

     

    Suresh

    • Hello Suresh,

      usually with the Tenant Administrator role you would have the authorization to delete queues in the queue monitor.

      BR,

      Mandy