Skip to Content

This blog describes how to use the new XI receiver adapter, which will be available for customers with the 10-June-2018 release. It describes the configuration options, size limits and monitoring of your scenario.

Configuring a Scenario Using the XI Receiver Adapter

Many customers run existing on-premise backend systems that can connect to an integration broker using the XI 3.0 protocol . In most cases, the broker is an SAP Netweaver PI (Process Integration) or PO (Process Orchestration) system. Now, with the new XI adapter, customers can also connect their on-premise backends to SAP Cloud Platform Integration using the XI 3.0 protocol. Whereas the XI sender adapter can be used to receive messages via the XI 3.0 protocol, the XI receiver adapter can be used to send out messages using the XI 3.0 protocol.

In this blog, I focus on the XI receiver adapter and will create a sample scenario for the XI receiver adapter. In this scenario an asynchronous request is sent to a backend using the XI 3.0 protocol. In blog Configure Scenario Using the XI Sender Adapter I will continue with the end-to-end scenario configuring the XI sender adapter to receive an asynchronous response.

To send a message via the XI 3.0 protocol in Cloud Integration you need to configure an integration flow with an XI receiver adapter and you need to configure the receiver backend to receive the message.

Configure the Receiver System to Receive the XI Message from Cloud Integration

The first step is to configure the receiver backend to receive a message via the XI 3.0 protocol. You need to check that the local integration engine is configured.

Configure Local Integration Engine

Configure the Lokal Integration Engine in the backend using transaction SXMB_ADM -> Integration Engine Configuration. Here the Role of the Business System defined should be Application System.

Construct XI Engine URL

Then you need to construct the URL under which the local integration engine of the application system can be called. This URL needs to contain the HTTPS port the system exposes. Check in transaction SMICM -> Goto -> Services. Using the host name and port of the HTTPS service construct the URL using the following pattern:

https://<host name>:<port>/sap/xi/engine?type=receiver&sap-client=<client>

Get Name of the Business System

To be able to configure the XI receiver channel you need to know the name of the business system of the applications system. The easiest way to find out this information is to use transaction SLDCHECK in the application backend. In section LCR_GET_OWN_BUSINESS_SYSTEM you find the id of the business system. Usually it has the pattern: <SID>_<client>.

Check Available Interfaces

Use transaction SPROXY to find out which XI services are available in the system. For our sample we use the services of the PI flight booking demo examples. Check for the Service Interface FlightBookingOrderRequest_In in namespace http://sap.com/xi/XI/Demo/Airline.

Check that the interface is active. This interface will process the request and send back a asynchronous response (interface FlightBookingOrderConfirmation_Out) to Cloud Integration. The configuration for the response processing is described in the blog Configure Scenario Using the XI Sender Adapter and can be done as soon as the request configuration is successfully finished.

If this interface is not available in your backend, you can also use another Inbound interface for the scenario.

Now we have all the details to configure the integration flow in the Cloud Integration tenant.

Configure the Integration Flow Sending the XI Message

First, we configure the integration flow in the Design section of the Web UI. Create an integration flow, connect the end message event with the receiver participant and select the XI adapter.

Configure the XI Receiver Channel

First, you configure the Connection Details in the Connection tab in the XI receiver channel.

  • Configure the Address field with the XI Engine URL constructed in one of the last steps.
  • Select the Proxy Type you want to use to connect to the backend. On-Premise indicates that in our sample scenario we use the cloud connector configured with the Location ID CC2 to connect to the on-premise backend. The configuration of the cloud connector will not be described here. If the cloud connector is required to connect to the backend refer to the SAP documentation for the cloud connector.
  • Select the Authentication Type required, in this sample scenario we use Basic Authentication and so have to enter the Credential Name as well. Make sure that you deploy the credentials on the tenant before you deploy the integration flow.

In addition to the connection details, you need to define the XI-specific configuration:

  • Define the XI Identifiers For Sender system. We define a name for the Sender Communication Component. This name is used in the backend to identify the sender of the message.
  • Define the XI Identifiers For Receiver. We define the name for the Receiver Communication Component. Set the business system of the receiver backend here, use the name you retrieved in one of the last steps using the SLDCHECK.
  • As Service Interface define the interface you want to call in the backend, for our scenario we use FlightBookingOrderRequest_In.
  • As Service Interface Namespace define the namespace of the interface you want to call in the backend, for our scenario we use http://sap.com/xi/XI/Demo/Airline.

In the Delivery Assurance tab you specify the Quality of Service and, in case of Exactly Once, the Retry configuration settings. Let’s check the options in more detail.

To define the Quality of Service, you have the following options: either Exactly Once or Best Effort:

  • Best Effort is used for synchronous request-reply scenarios, that means, in case your sender system has a synchronous interface through which you like to send a message to Cloud Integration you use this option. If you select this option, you don’t need to configure any additional settings because the message request and the response are processed immediately in Cloud Integration. Therefore, no temporary storage of the message for later retry is involved.
  • Exactly Once is used for asynchronous one-way scenarios, that means if your sender system has an asynchronous interface through which you like to send a message to Cloud Integration you need to use this option. If you configure this option the message is temporarily stored in the Cloud Integration tenant and, if an error occurs, the message is retried from there. The sender gets the successfully received status as soon as the message is persisted in the temporary storage.

We configure Exactly Once for our sample scenario as we want to send an asynchronous message. In this case the Temporary Storage needs to be defined. You can select to store the message temporarily either in the Data Store or in JMS Queues:

  • When Data Store is selected, the message is persisted temporarily in the tenant’s database. The stored message can be monitored in the data store monitor.
  • If JMS Queue is selected, the message is stored temporarily in a JMS queue on the connected JMS broker. Note, that the JMS Queue option is more performant, however, it is only available if you have Enterprise Messaging purchased for the tenant. (More details can be found in the in blog ‘How to activate Message Broker’.)

If Data Store is configured as Temporary Storage, the following additional settings need to be defined:

  • Retry Interval: Here you define after which time a retry shall be triggered if there was an error during message processing.
  • Exponential Backoff: If the flag is selected (which I would recommend), the retry interval is doubled after each unsuccessful retry.
  • Maximum Retry Interval: Here you enter the limit to avoid endless increase of the retry interval caused by the exponential backoff setting. This field is only available is Exponential Backoff is selected.
  • Encrypt Message During Persistence: If this option is selected the message is encrypted in the data store during temporary storage. Note that this is recommended if the message contains sensitive data. However, the drawback is that this setting reduces the performance slightly.
  • Note that no configuration option is available for dead-letter queue handling. If the data store is used as temporary storage, the out-of-memory handling is always active. This means that after the 3rd retry caused by an out-of-memory, a message is blocked and taken out of further processing.

If JMS Queue is configured as Temporary Storage the following additional settings need to be defined:

  • Retry Interval: Here you define after which time a retry shall be triggered if there was an error during message processing.
  • Exponential Backoff: If the flag is selected, which I would recommend, the retry interval is doubled after each unsuccessful retry.
  • Maximum Retry Interval: Here you enter the limit to avoid endless increase of the retry interval caused by the exponential backoff setting. This field is only available is Exponential Backoff is selected.
  • Dead-Letter Queue: You should select this option to handle potential out-of-memory situations caused by large messages. If selected,  a message is blocked and taken out of further processing after the 3rd retry caused by an out-of-memory. More details about the dead letter queue handling can be found in blog How to configure Dead Letter Handling in JMS Adapter.
  • Encrypt Message During Persistence: If this option is selected, the message is encrypted in the JMS queue during temporary storage. Note that this is recommended if the message contains sensitive data. However, the drawback is that this setting reduces the performance slightly

Dynamic Configuration in XI Receiver (Supported with the 5th-August-2018 update)

Most fields in the XI Receiver adapter are supported for dynamic configuration. This means you can configure them from a header or property:

The following fields can be configured via header or property:

  • Address
  • Location ID
  • Credential Name
  • Private Key Alias
  • Sender Communication Party
  • Sender Communication Component
  • Receiver Communication Party
  • Receiver Communication Component
  • Receiver Service Interface
  • Receiver Service Interface Namespace

Note that you need at least version 1.6 of the XI Receiver channel. This is the version that is available with the 5th-August-2018 update.

Configure Transaction Handling

You need to configure the correct transaction manager in the integration process for transactional end-to-end processing. Our process does only contain one direct XI Receiver without splitter or sequential multicast; so we don’t need a specific transaction handler. Select the integration process and switch to the Processing tab. In Transaction Handling drop down select Not Required.

More details about the different transaction handling options and existing limitations are described in the blog ‘How to Configure Transaction Handling in Integration Flow’.

Configure the Inbound Channel

Configure the integration flow with the inbound channel required by your scenario. For the sample scenario we use the SOAP adapter and trigger the message from SOAP UI. But you can use any other adapter supporting asynchronous requests as well.

  • Define the Address the SOAP endpoint will expose.
  • Select One-Way as Message Exchange Pattern because the scenario is an asynchronous one-way scenario.
  • Select Robust as Processing Setting to receive errors back that may occur during message processing.
  • Define the Authorization you want to use. As we want to connect using user and password, we use User Role with the default User Role ESBMessaging.send.

Deploy the Integration Flow

Now you can deploy the integration flow. Afterwards check that the integration flow was started successfully in the Manage Integration Content monitor.

Get the Endpoint URL to be called

To be able to call the integration flow from the SOAP UI you need to know the endpoint URL to be used. For this purpose, check the Endpoints tab in the Manage Integration Content monitor. There the URL is shown and can be copied to the clipboard using the Copy option.

Sending a Message to the Cloud Integration Endpoint

Now you can run the scenario. Use the SOAP UI to send the call. You can use the following request payload:

<soapenv:Envelope xmlns:soapenv=”http://schemas.xmlsoap.org/soap/envelope/” xmlns:dis=”http://camel.apache.org/cxf/jaxws/dispatch”>
<soapenv:Header/>
<soapenv:Body>
<ns1:FlightBookingOrderRequest xmlns:ns1=’http://sap.com/xi/XI/Demo/Airline’>
<AgencyData>
<AgencyID>00000110</AgencyID>
<OrderNumber>00000001</OrderNumber>
<ItemNumber>01</ItemNumber>
<OrderType>Single</OrderType>
</AgencyData>
<FlightID>
<AirlineID>LH</AirlineID>
<ConnectionID>401</ConnectionID>
<FlightDate>2018-04-01</FlightDate>
</FlightID>
<ClassCode>Y</ClassCode>
<PassengerName>Max Muster</PassengerName>
<PassengerBirthdate>2000-12-31</PassengerBirthdate>
</ns1:FlightBookingOrderRequest>
</soapenv:Body>
</soapenv:Envelope>

After triggering the request from SoapUI, you can check the execution in Cloud Integration’s monitoring and in transaction SXMB_MONI in the backend. In Cloud Integration’s monitoring you will see that two messages were processed; one message for storing the message in the temporary storage and one for consuming it from there and further processing it. More details about the monitoring in Cloud Integration can be found below in the Monitoring section.

In transaction SXMB_MONI you should see two messages: the inbound request from Cloud Integration which should be processed successfully and an outbound request trying to send the response. But as we have not yet configured the response processing this message will probably have the status failed. The response processing will be configured in the blog Configure Scenario Using the XI Sender Adapter.

 

Monitoring

To check the processing in Cloud Integration detail, you can use the monitoring tools provided by Cloud Integration.

Message Monitoring

The message processing can be checked in detail in the section Monitor Message Processing. Select the message you sent and check the processing status. You should find two message processing logs, one for the inbound messaging that saves the message to the data store/JMS queue, and one for processing the message from the data store/JMS queue and sending it to the receiver system. Those two messages are correlated via the CorrelationID available in the textual representation of the message processing log.

With the 8th-July-2018 update of Cloud Integration, this Correlation ID is in addition shown in the Properties of a message processing log and you can also search for all messages having the same Correlation ID using the ID search field at the top of the monitor:

In the event of a failure when sending to the receiver system, the outbound message will go into status Retry. The message will then be retried as configured in the XI receiver channel. The detailed error information is available under the link Message Processing Log.

Data Store Monitor (For Messages Stored in Data Store)

The messages that are stored temporarily during processing can be seen in the Data Stores tile in the Manage Stores section in the monitoring dashboard. During successful processing, the messages are removed immediately after the processing was completed. In case of an error the messages are kept in the data store to trigger the retry from there.

The used data store name is auto-generated according to the following pattern: <participant name>_< XI channel name>. Below the data store name <integration flow name>/XI is shown. In the monitor all messages are listed with a Status. Two statuses exist:

  • Status Waiting means that the message waits for consumption.
  • Status Overdue indicates that the message was not consumed and processed in the due time, which is 2 days in case of the XI adapter.

In the monitor the Message ID is shown as a link. If you select the link the respective message processing log is opened in the message monitor.

Note that data stores that do not have messages persisted at the time the monitor is opened are not shown in the monitor. This is because data stores are created “on the fly” during message processing if required and removed after the last message was removed from the data store.

Lock Monitor (For Messages Stored in Data Store)

In case the message was involved in several node outages, for example caused by an out-of-memory, the message gets locked in the Lock Monitor. Those messages are not retried any longer. Notice that you will probably not see any entry in this monitor during testing your sample scenario, as this entry will only be permanently available in case of a runtime node outage. So, see this section more as additional information.

The entry will be generated with the following parameters:

  • Component: XI
  • Source: XI_<integration flow name>.<participant name>_<channel name>
  • Entry ID: corresponds to the ID shown in data store monitor

You need to check if the message was the root cause of the error, maybe it is just too large to be processed. Then you need to ask the sender to send the data in smaller messages. Afterwards you can delete the message in the Data Store monitor and then also delete the lock in the Manage Locks monitor. If you are confident that the message did not cause the outage you can release the lock in the Manage Locks monitor by using the Release button.

Queue Monitor (For Messages Stored in JMS Queue)

The messages that are stored temporarily during processing can be seen in the Message Queues tile in the Manage Stores section in the monitoring dashboard. During successful processing the messages are removed immediately after the processing was completed. In case of an error the messages are kept in the JMS queue to trigger the retry from there.

The used JMS queue name is auto-generated according to the following pattern: XI.<Integration Flow Name>.<Channel Name>.<Guid>. In the monitor all messages are listed with a status. Four statuses exist:

  • Status Waiting means that the message waits for consumption.
  • Status Failed indicates that the last processing or retry of the message ended with an error.
  • Status Blocked comes into play if dead-letter queue is configured in XI channel and if the message was involved in several node outages and so is taken out of processing. See dead-letter queue handling above.
  • Status Overdue indicates that the message was not consumed and processed in the due time, which is 2 days in case of the XI adapter.

In the monitor the Message ID is shown as a link. If you select the link the respective message processing log is opened in the message monitor.

You can see the number of retries that have already been executed and the time of the next retry. You can also trigger an immediate retry of the message or delete specific messages if you no longer want them to be processed.

Queue Deletion

If you do no longer need a specific queue (for example if you un-deploy the integration flow), you have to delete the queue manually in the Queue Monitor if the queue contains messages. The queue cannot be deleted automatically because this could cause data loss. Only the scenario owner knows if the messages can be deleted or are still required.

To find unused queues in the monitor you may use the checks described in the blog ‘Checks in JMS Message Queue Monitor’.

JMS Resource and Size Limits

If you are using JMS for temporary storage you need to keep in mind that the connected JMS messaging instance, that is used has limited resources. The Enterprise or Messaging License sets a limit on the queues, storage and connections that you can use in the messaging instance. The limits are described in more detail in the blog ‘JMS Resource and Size Limits‘.

Important Considerations and Limitations

When using the XI sender or receiver adapter, there are some very important facts you need to be aware of:

Changing Storage Between Data Store and JMS May Cause Data Loss:

As the adapter stores messages for retry in case of an error either in a JMS queue or in the Data Store, you need to keep in mind that changing this setting for an already running scenario may cause loss of data. This is because there can still be messages in the old storage, which will not be retried anymore after the storage changed. Those messages would stay in the data store or JMS queue until they are deleted (see field Retain Until in Data Store or Queue monitor).

Because of this be very careful with such changed in productive scenarios. Make sure there are no messages left in the ‘old’ storage before changing this setting.

Changing Participant or Channel Name May Cause Data Loss:

As the data store or JMS queue name for retry is generated based on the participant and channel name, a new data store or JMS queue would be generated if you change one of those names. As there can still be messages in the old storage, these messages will not be retried anymore after the storage changed. Those messages would stay in the data store or JMS queue until they are deleted (see field Retain Until in Data Store or Queue monitor).

Because of this be very careful with such changes in productive scenarios. Make sure there are no message left in the ‘old’ storage before changing participant or channel name. With the 30-September-2018 update you have the option to move all messages from one JMS queue to another, if required. This option would help you in case you urgently need to change the channel or participant name or if you have done it without knowing the constraint that a new queue would be generated.

To move messages from one JMS queue to another use the Message Queues monitor. In the single line actions for the ‘old’ queue select the Move action:

In the Move Messages dialog select the newly generated queue and choose Move. All messages of the old queue are now moved to the new one.

Note that there is currently no such move option for the data store. The blog will be updated as soon as this is available.

 

Wrong transaction handling configuration may cause inconsistent data and/or data loss:

The two storage options, JMS Queue and Data Store, require different transaction handlers. Because of this it is crucial that you define the correct transaction handler:

  • JMS Queue: If a JMS Queue is used for temporary storage in the XI receiver adapter and the XI adapter is used in a sequential multicast or splitter pattern, you need to configure the Transaction Handling as Required for JMS.
  • Data Store: If the Data Store is used for temporary storage in the XI receiver adapter and the XI adapter is used in a sequential multicast or splitter pattern, you need to configure the Transaction Handling as Required for JDBC.
  • For the XI sender adapter no transaction handler is required.
  • For the XI receiver adapter no transaction handler is required if the XI adapter is not used in a sequential multicast or in a split scenario.
  • As there is no distributed transaction support in Cloud Integration you cannot combine JMS and JDBC transactions. This means that you cannot ensure transactional behavior in scenarios using the XI receiver adapter with JMS storage in multicast scenarios together with flow steps that need a JDBC transaction handler, like for example Data Store or Write Variables.

More details about the transaction handling configuration and the constraints in configuration can be found in blog ‘How to configure Transaction Handling in Integration Flow‘.

The XI adapter currently has some limitations:

  • No explicit retry handling via Retry Header in Exception Sub-Process possible possible yet.- Solved with the 2nd-September-2018 update, see blog Configuring Explicit Retry in Exception Sub-Process for XI Adapter Scenarios.
  • XI receiver adapter is currently neither allowed in Request-Reply nor in Send step.
  • MPL status stays in Retry even if the message is deleted from JMS queue or data store. Nevertheless in runtime no retry is executed anymore.
  • Only AtLeastOnce (ALO) is ensured by the XI adapter, no ExactlyOnce (EO). That means if the sender system sends the same message multiple times, the XI adapter forwards those multiple messages.
  • Acknowledgements are not supported
  • Attachments are not supported
  • ExactlyOnceInOrder (EOIO) is not supported.

The blog will be updated regularly with the new features.

To report this post you need to login first.

19 Comments

You must be Logged on to comment or reply to a post.

  1. Eng Swee Yeoh

    Hi Mandy

     

    Thank you for sharing on this much awaited functionality.

     

    I would like to ask about the JMS Queue naming – XI.<Integration Flow Name>.<Channel Name>.<Guid>

     

    Can you explain further how <Guid> is derived – is it at per message level, per channel, or some other granularity? I would like to know because your other blog mentions that the maximum queue is 30, so this might mean that there is a limit on how many XI flows/channels/messages/etc are possible in a single CPI tenant.

     

    I also noticed that the retry functionality (for Data Store) seems to go on forever irrespective whether Exponential Backoff is selected (with maximum retry interval set) or not. I had to manually remove the Data Store entries to stop the retries.

     

    Regards

    Eng Swee

    (0) 
    1. Mandy Krimmel Post author

       

      Hello Eng Swee,

      the Guid is auto-generated based on the combination of iFlow/participant and channel. This is definitely unique and cannot be re-used between flows. And yes, this goes into the count of queues, which is limited to 30 queues. (but in future additional JMS resources can be bought in addition).

      The retry is always going on forever, this has nothing to do with the exponential backoff. The exponential backoff only restricts the max time interval between the retries. For an explicit modeling of max retries you need to use an exception sub-process and model this on your own. But this is currently not supported for the XI adapter, this will come in one of the next updates. The blog will be updated then.

      BR,

      Mandy

       

      (0) 
      1. Eng Swee Yeoh

        Hi Mandy

         

        Thanks for the quick response and explanations!

         

        So can I say that at the moment, we can only have roughly 30 integration flows with XI receiver channels?

         

        If the retry goes on forever, that’s gonna make the XI adapter pretty much useless for productive scenarios at the moment.

        (0) 
        1. Mandy Krimmel Post author

           

          Hi,

          Yes, this is true. With 30 receiver channels using JMS as temp storage you are at the limit.

          The retry configuration option will be added shortly (in one of the next updates), we know that this is urgently required.

          BR,

          Mandy

          (0) 
          1. Eng Swee Yeoh

            Oops.. sorry, one last question (for now) –

             

            Does the 30 JMS queues limit holds true as well when running the CPI content on on-premise PO 7.5?

            (0) 
            1. Mandy Krimmel Post author

               

              XI adapter is not released yet for IGW profile. And if it gets released the XI adapter will most probably be available with data store option only.

              BR,

              Mandy

              (0) 
    1. Mandy Krimmel Post author

       

      yes, if you select Best Effort in the XI receiver channel you are able to execute synchronous calls (as mentioned in the blog).

      BR,

      Mandy

      (0) 
        1. Mandy Krimmel Post author

           

          Hi,

          as clearly mentioned in the limitations section, XI adapter is currently neither allowed in Request-Reply, nor in Send step. It’s planned, but not yet supported.

          BR,

          Mandy

          (0) 
  2. Frank Bakermans

    Hi Mandy,

    Thanks for you blog. I am using the CPI XI adapter to call an ABAP Proxy on a SAP backend system. The message has been received by the backend system, but is not routed to the proxy, but rather to the integration server. The Pipeline it is classified as SENDER rather than RECEIVER.(if I compare with similar message, coming from a SAP PI system, calling the same ABAP Proxy)

    In the logging I have found the following error:

    Error while receiving by HTTP (error code: 200, error text: Error Parsing Response. No XI Response Received.)

    Does this ring any bells?

    Thanks,

    Frank

    (0) 
    1. Mandy Krimmel Post author

       

      Hi Frank,

      looks like some configuration problem.

      Is it a synchronous interface you are calling? Are you calling this with setting BE in Delivery Assurance Tab in XI channel?

      BR,

      Mandy

      (0) 
        1. Mandy Krimmel Post author

           

          Hi Frank,

          it’s hard to say via remote analysis. Could you maybe open a ticket on LOD-HCI-PI-RT?

          Please attach the integration flow project and the mpl with log level Debug and the complete log from the backend (from sxmb_moni)

          Best regards,

          Mandy.

           

          (0) 
        2. Mandy Krimmel Post author

           

          One more idea: what is the endpoint URL configured in the XI receiver channel? Does it have URL parameter type=receiver?

          https://<host name>:<port>/sap/xi/engine?type=receiver&sap-client=<client>

          (0) 
  3. Yan Rakhorst

    An additional remark:

    It’s possible to get the following error message when trying to send a message from CPI to a SAP ECC backend using the XI receiver adapter:

    org.apache.cxf.transport.http.HTTPException: HTTP response ‘403: Forbidden’ when communicating with http://<host&gt;:<port>/sap/xi/engine?type=receiver&sap-client=<client>

    In that case you might need to check your access control table in your local SAP Cloud Connector settings. Check SAP note 2473005 and the SAP help page (section ‘Limiting the Accessible Services for HTTP(S)’) for further information.

    (0) 

Leave a Reply