Skip to Content
Technical Articles

EIPinCPI – Pipes-and-Filters

Previous – Format Indicator | Index | Next – Message Router

This week we begin studying Message Routing patterns. First Message Routing pattern is Pipes-and-Filters.

When do I use this pattern?

Use this pattern when the steps for the integration are reusable. The simplest example is reusing a flow that connects to the receiver system. For example, consider integrating a CRM system with ECC. The CRM system sends Customer and Orders to the ECC system. Here, the flow for both Customer and Order consists of 3 steps:

  1. Read from CRM System
  2. Map to IDoc
  3. Send IDoc to ECC

Reading the orders and customers would require separate flows. Mapping will also require separate flows as the source and target messages are different. However, sending IDoc to ECC can be designed to be a reusable step.

Pipes-and-Filters in CPI

In CPI, I’ll use JMS Adapters to demonstrate the Pipes-and-Filters pattern.

Demo of Pipes-and-Filters – Customer Integration

Let’s consider Northwind as the CRM system that creates/updates Customers. The flow will read customers from Northwind, map them to DEBMAS IDoc, and send the DEBMAS IDoc to ECC.

Step 1: Read Customer

Here, OData Adapter is used to read Customers from Northwind and JMS Receiver Adapter is used to send the read customers into ‘Customers’ queue.

Other Patterns

Here, the Read Customer demonstrates Request-Reply pattern.

Step 2: Map Customer

Here, customers are read from the ‘Customers’ queue using JMS Sender Adapter, mapped to DEBMAS IDoc and put into ‘ECC’ queue.

Step 3: Send to ECC

This flow reads from ‘ECC’ queue and posts the IDoc to ECC.

Demo of Reusability in Pipes-and-Filters – Order Integration

When sending orders from the CRM system into ECC, the same 3 steps are applicable. Read orders from Northwind, map them to ORDERS IDoc, and send the ORDERS Idoc to ECC. However, the 3rd step of sending the IDoc to ECC is a reusable component.

Step 1: Read Order

Similar to Customer Integration, this flow reads orders and puts them into the ‘Orders’ queue.

Step 2: Map Order

Similar to Customer Integration, this flow reads orders from the ‘Orders’ queue, maps them to ORDERS IDoc, and puts them in the ‘ECC’ queue.

Step 3: Send to ECC

This is a common step between Customer integration and Order integration. It reads from ‘ECC’ queue and sends the IDocs to ECC.

Monitoring tip for Pipes-and-Filter pattern

Dividing the flow of a message into several components like this does not make message monitoring difficult. Use the SAP_ApplicationID header to effectively understand the flow of the message. Check the 5th tip in Eng Swee‘s tips for CPI development for more information. For official documentation, search for SAP_ApplicationID in Headers and Exchange Properties provided by the Integration Framework.

Conclusion

Pipes-and-Filters is a powerful pattern that promotes creating small components that are reusable. The dependency between these components is reduced through asynchronous processing. The Asynchronous processing is implemented by the use of JMS queues in CPI.

References/Further Readings

Hope this helps,
Bala

Previous – Format Indicator | Index | Next – Message Router

3 Comments
You must be Logged on to comment or reply to a post.
  • Good one again.

    I did not know this name, but I guess something that I have used often.

    I guess the JMS can be replaced by process direct in many cases, depending on what you want to achieve with it.

    • Thanks Daniel Graversen.

      I was also happy to have a name for the pattern so commonly used in CPI.

      When strictly following this pattern, I wouldn’t use the ProcessDirect between flows because ProcessDirect in CPI does not support asynchronous processing. The reusable steps should be able to process the message without depending on any other step at runtime.

      I believe Camel’s Direct VM component is underneath ProcessDirect. Camel’s Direct VM component has a query parameter named ‘synchronous’ to set whether synchronous processing should be strictly used, or Camel is allowed to use asynchronous processing (if supported) [Source: https://camel.apache.org/components/latest/direct-vm-component.html#_query_parameters_11_parameters). This is something for CPI to adopt from Camel. 🙂

  • Decompose a task that performs complex processing into a series of separate elements that can be reused. This can improve performance, scalability, and reusability by allowing task elements that perform the processing to be deployed and scaled independently.

    Walgreenslistens