Modularize PI Integrations via REST API's

[harsh_chawla]

Introduction

SAP PI is a mature integration platform with a predefined pipeline that allows organizations to deploy thousands of iFlows in a supportable manner. However, when building complex scenarios the pipeline’s limitations can lead to suboptimal iFlow designs. In this blog, I will:

1. Provide an overview of the standard PI pipeline


2. Elaborate of the limitations of the standard pipeline when building complex integrations


3. Propose a REST based API architecture that addresses these limitations

PI Pipeline Overview

PI Standard Pipeline

The PI java stack pipeline follows the above mentioned 5 steps.

1. Sender Agreement


2. Receiver Determination


3. Interface Determination


4. Message Mapping


5. Receiver Agreement

 

To understand this better let's map this pipeline to a manufacturing interface. Let's take an example of SAP ERP sending Process Orders to a Plant system.

PI Pipeline for Process Orders(single plant)

This would translate to the pipeline steps as follows:

1. Send Process Order from SAP to PI


2. Identify Destination Plant


3. Determine Plant Specific Interface


4. Transform data from SAP Format to Plant Format


5. Send Process Order data from PI to Plant 

The standard pipeline has some level of flexibility. The developer can add additional receivers or interfaces. For eg, if we wanted leverage a common SAP extract to send Process Orders to 100 plants, the pipeline would look as follows:

PI Pipeline for Process Orders(multiple plants)

Challenges

While the above example is technically feasible, it comes with a host of operational challenges when deployed as-is. For eg:

1. Complex Design: With an integrated design we end up with a single high complexity iFlow that needs to be edited 100s of times as we roll out the process order integration to each plant.


2. Mapping: We only have one place per plant to run an operational map. For eg: We are unable to run a common generic map, followed by a plant specific map.


3. Limited Visibility: As all messages flow through the same iFlow, the support teams have limited visibility into the integration via the Message Monitoring screens. This can make iFlows with high message volume or high receiver counts challenging to support.


4. Traffic Control: In case of plant specific outages and maintenance windows, there is no way to hold/queue messages for a specific plan without impacting the entire iFlow.

 

Improved Architecture using REST URL scheme

PI REST Pipeline(one plant)

 

We can address these challenges by unbundling this interface into two modular iFlows that communicate via a REST API. The Generic iFlow accepts process orders from SAP and performs plant agnostic operations on the payload. It then forwards on the message payload to a REST api via a dynamic URL scheme as listed above. The Plant Specific iFlow accepts messages by listening on its REST API URL and then executes Plant specific transformations and protocol interactions.

In my experience, the following dynamic REST URL scheme works well for connecting the two iFlows: 

https://host:port/RestAdapter/<Plant#>/<DocType>;

Eg:

https://myPI/RestAdapter/Plant1/ProcessOrder

https://myPI/RestAdapter/Plant2/ProcessOrder

 

This architecture scales out seamlessly for 100 plants as follows:

PI REST Pipeline(multiple plants)

 

Benefits of REST API based PI architecture

1. Modular Design: By unbundling the one giant integrated iFlow into a modular design, we reduce overall complexity of the integration. This allows us to incrementally roll out the integration to hundreds of plants without impacting any existing plants.


2. Reusable Mapping: The generic iFlow allows us to leverage a common map across all plants that can perform canonical operations that are independent of plant requirements.


3. Better Monitoring Visibility: By breaking out traffic flow into plant specific iFlows, we give the support team direct visibility of plant specific messages via the standard PI monitoring screens


4. Traffic Control: With the new design, the PI team has the ability to regulate (or prioritize) traffic to individual plants without affecting the whole interface.

Summary

After having worked on PI for almost two decades, I've come to the conclusion that by limiting the capabilities of the standard PI pipeline, the SAP product team made a useful trade off between flexibility and simplicity. In most cases the standard pipeline is good enough, in that it allows developers to build stable integrations that are easy to support. However, in the few exception scenarios where the integration team is challenged to "do more" in PI, this REST based extension framework allows developers to build more supportable and scalable integrations. I hope you find this information useful. 

How do you modularize your complex PI integrations?