Using RabbitMQ/AMQP for high volume integration with Hybris Commerce Solution.
A customer has very specific requirement for Hybris integration with SAP ECC thru their SAP PI system. The process should be able to handle thousands of orders per second from Hybris at peek business periods.
This blog describes an architecture with RabbitMQ as an intermediate buffer system to absorb peek transaction volume and the Advantco AMQP adapter for SAP PI/PO integration.
Hybris integration options.
Hybris provides these three main options for integration with external systems:
- SOAP/RESTful web services
- Java Message Service (JMS)
- Spring Integration
From SAP PI perspective, the SOAP/RESTful web services and the JMS options are the most obvious options as these are standard support by SAP PI but each of these options have their limitation.
SOAP/RESTful web services option: Hybris provides a RESTful API for exporting business data in JSON format. Our customer used the Advantco REST adapter to expose a RESTful service on the SAP PI system to receive incoming requests from Hybris. Although they noticed significant performance improvements comparing to the classic SOAP/XML services, their SAP PI system was not able to handle the thousands of requests per second due to the exceptional high number of threads that must be available to process these requests. This exercise did provided a conclusion that Hybris REST/JSON is the correct way to export data as it is very fast.
Java Message Service (JMS) option: SAP PI does provide a standard JMS adapter but it does not have a native JSON/XML converter.
The chosen architecture option.
To prevent flooding their SAP PI with RESTful requests, the customer decides for a buffer system to absorb the incoming requests and have their SAP PI system pulls the data from the buffer system. RabbitMQ was the ideal solution as it is a very lightweight broker which can handle huge volume of data and supports native JSON format.
To pull data from RabbitMQ, the customer uses the Advantco AMQP adapter which is fully compatible with RabbitMQ and has the support for JSON/XML conversion. The number of application threads on the SAP PI system determines how much fast data can be fetch from RabbitMQ. This scenario is different from the RESTful scenario in that the SAP PI system has a control on how much data it want to process at anytime, independent of the data volume coming from Hybris.
This architecture setup satisfy the requirements to handle thousands of orders from Hybris while preventing their SAP PI to crash because of the sheer amounts of data.
RabbitMQ : http://www.rabbitmq.com/
Hybris : https://www.hybris.com/en/
AMQP adapter : https://www.advantco.com/product/adapter/amqp
AMQP : https://www.amqp.org/