XI/PI File and JDBC Receiver Adapter performance a...

kenny_scott · ‎12-05-2008

To help explain the Receiver Adapter performance and availability improvements, I have outlined below, the normal thread activity for processing 'classical scenario' ** XI messages sent from the Integration Engine to the Adapter Framework Messaging system,

i.e.,

message is persisted, i.e., put into a receiver messaging system queue
a worker thread (also referred to as a consumer thread) is requested from the thread pool
a worker thread is removed from the thread pool
a message is assigned to the thread
the thread will perform the message processing from this point
once the processing is successfully completed the thread will return to the pool

The recent improvements for Receiver Adapters (File & JDBC specifically) are:

(1)

The introduction of queue separation (i.e., separate adapter type sets) delivered with >= XI 3.0 SP19 / 7.0 SP11 improved the overall performance and robustness of the Adapter Framework Messaging System and SAP JCA Adapters.

In summary, each adapter type (e.g. File, JDBC etc ..) now has 4 queues,

i.e.,

Directions: Outbound, Inbound

Mode: Synchronous, Asynchronous

Each queue has it's own dedicated Thread Pool and each Thread Pool has a maximum capacity. This consumer thread configuration is done in the JEE service: SAP XI AF Core, parameter -

messaging.ConnectionDefinition

The default value for each queue equals the settings in the global entry, i.e., the first entry in the parameter string. Different consumer thread values can be defined per adapter type by adding specific adapter sets,

e.g.,

see the SDN blog

Messaging System queue properties after XI 3.0 SP19 / XI 7.0SP11

(2)

Some adapters though were unable to handle requests in parallel, e.g., the file adapter processes one write/append operation to the file system. Once the file is successfully written the thread is released and the next ‘file receiver adapter' thread processes the next file request.

These adapters, by default, work in a sequential manner; only one thread is active at a time.

Although work around methods existed to enable File and JDBC Receiver Adapter parallelism (e.g., create additional File Receiver adapters (different name) with respective additional receiver determination and agreements) it is since XI 3.0 SP20/PI 7.0 SP12 and the introduction of the JDBC and File Receiver adapter Communication Channel parameter -

Maximum Concurrency

- that parallelism has been delivered for these adapters.

The parameter value specifies how many messages the adapter processes in parallel in a cluster node. The default (old behaviour) is 1 but to enable parallelism this value should be set to a value >1.

(3)

To improve the availability of Receiver Adapters' (i.e., prevent problem channels halting all the instances of that Adapter type) the parameter -

queueParallelism.maxReceivers

- has been introduced in XI 3.0 SP20 / 7.0 SPS12.

See the CSS note -

#1136790 Blocking receiver channel may affect the whole adapter type

and the SAP online help (Properties Related to the Messaging System) description of the property -

messaging.system.queueParallelism.maxReceivers

Value type: Integer

Default: 0 (disabled)

This property specifies the maximum number of parallel worker threads per queue, based on the receiver fields of the processed messages. A value of 2, for example, will limit the amount of parallel worker threads for one receiver channel instance to 2, allowing the remaining configured worker threads to process messages for other receiver channel instances of the same adapter type.

This globally defined parameter prevents situations, e.g., where worker threads continue to pick up messages from a growing Messaging System queue for delivery to a receiving backend system that has stopped processing messages (e.g., due to network error etc..)

Eventually all the available consumer threads for that adapter queue would have messages assigned to them that they can not deliver. These messages remain in a limbo status and the threads are not returned to the pool.

By implementing the note -

#1136474 XI 3.0/7.0: Setting Timeout for acquiring DB/FTP resources

- messages will not remain in limbo but will - after a configurable timeout value has been reached - be returned to the Messaging System queue in retry mode if no connection is available.

For a further description of this timeout and the maximum concurrency channel parameter refer to the CSS notes:

#831162 FAQ: XI 3.0 / PI 7.0 / PI 7.1 JDBC Adapter

Q28. Effect of setting Maximum Concurrency and poolWaitingTime

#821267 FAQ: XI 3.0 / PI 7.0/ PI 7.1 File Adapter

Q47. Effect of setting Maximum Concurrency and poolWaitingTime

-------------------

#1473299 Setting Maximum Concurrency and poolWaitingTime in File/Jdbc

#1604091 Messages delivered by File/JDBC receiver stuck in DLNG state

=============

**

For Integrated Configuration Object (ICO) scenarios see the SDN blog

Tuning the PI/PO Messaging System Queues