New Feature in Integration Advisor: Reordering of Source Structure

Introduction

If you are familiar with doing complex mappings in SAP Integration Suite, it might be of interest for you, that Integration Advisor Capability of SAP Integration Suite now introduces a new powerful feature called pre-transformation of source Message Implementation Guide structure. With this new feature Integration Advisor now supports the reordering of source structure before the actual mapping is taking place.

In certain integration scenarios it is inevitable to apply specific structural changes to the incoming data structure. This is usually the case if the source and target side structures differ that much, that a simple 1:1 mapping between hierarchy levels on both sides is not possible.To cope with such situations, Integration Advisor now allows to apply structure modifying operation prior to the actual mapping is taking place. This new and optional transformation is called Pre-transformation because at runtime it is applied to the message payload before the actual transformation is applied and it is applied to the Message Implementation Guide at design time also before the mapping utilizes the transformed source structure.

By adding a Pre-transformation to the Mapping Guideline it is now possible to tailor the incoming data structure to fit the structural requirements of the outgoing data structure in integration scenarios where this is required. The result of the Pre-transformation then is used in the Mapping Guideline as the source side Message Implementation Guide while the original source side MIG becomes the input for the Pre-transformation. The entire functionality, that is required to develop and simulate Mapping Guidelines with Pre-transformation is encapsulated in the new version of the Mapping Guideline Editor in Integration Advisor.

The export of runtime artifacts for SAP Cloud Integration automatically contain all relevant entities for executing the Pre-transformation along with the main mapping. Hence, it is completely transparent to the executing integration flow whether an IA mapping contains a Pre-transformation or not. No changes or extensions to any integration flow are required for the Pre-transformation to be executed along with the main transformation. This way, Pre-transformation integrates perfectly with all existing integration flow, that might already be in use for running Integration Advisor mappings.

The first two operations

The first version of Pre-transformation in Integration Advisor comes with two operations. The aim of the provided Pre-transformation operations is to encapsulate complex structural changes into one single step whenever possible.

Further operations are already on the roadmap.

Copy Referenced Node

This operation allows to create a copy of a group node into a different group node, that is not the parent of this original group node. The copy operation itself is controlled by a reference between the copied node and the receiving group node. The user is free to choose that reference nodes. As a result, only such group nodes are copied into such receiving group nodes, where the two reference nodes contain the same value. Reference nodes always must be descendants of the involved group nodes (the copied node called “referenced node” and the “receiving node”).

For instance, a source structure contains Items and corresponding Handling Units as sibling. Though the target structure requires the Items to be children of the corresponding Handling Units. Whenever it is possible in the source structure to derive a reference between the corresponding instances of Items and Handlings Units for example because both contain an ID, that allows to link them together, the operation Copy Referenced Node can be used to copy the items into the correct Handling Units.

Group by Key

The purpose of this operation is to allow grouping of group nodes according to properties of these pre-existing group nodes. This property must be an {descendant} {{leaf}} node of the group that shall be grouped and is called the “group key node”. For all instances of a group with the identical value in group key node, a new supergroup node will be created carrying a distinct group key. This new group collects all instances of the original group with one specific value in group key node. As the result of this operation, the structure will contain one instance of the newly created supergroup for each distinct value of group key node. All instances of the original group node will be moved into the super group with the value of group key corresponding to the value of the group key node of this original group.

Assuming a source structure contains an arbitrary number of handling units of different types as sibling nodes on the same hierarchy level. Further on these units can appear in arbitrary order while the target structure required such units to be in order corresponding to their type. In this case the handling unit itself would be the group to be grouped into newly to be created instances of the super group. The type of the unit would become the Id to be used for grouping. The result of the operation is one instance of the supergroup for each distinct type of handling unit. All handling units with the corresponding type will be contained in their matching supergroup.

A simplified real world example

Initial situation is like shown in the image below. The shipment on the source side contains the handling unit elements as sibling nodes whereas the target side structure requires the items to be contained in their corresponding handling units. A reference between both types of nodes can be derived by correlating HandlingUnit/unitid to Item/handlingUnitId. Further on inside a handling unit different types of Items need to be grouped by their type.

Two different structures that cant directly be mapped to each other

The required correlation and the structural change can be applied by using Pre-Transformation operation copy referenced node.

To achieve this, a Pre-transformation needs to be added to the Mapping Guideline on the Overview tab of the Mapping Guideline. While in edit mode clicking on the plus sign near “Pretransformation” a new Pre-transformation is added to the Mapping Guideline.

Adding the new PTS

Right after clicking the add button, the Mapping Guideline editor switches to the newly added Pre-transformation showing the unchanged source structure on both sides.

After applying the operation copy referenced node to the Item Node and clicking on Execute, a copy of the original Item node is created as child of the handling unit node.

After simulation with a payload, that contains three handling units and three different Items  from which one belongs to the first, 2 belong to the second and none belongs to the third handling unit, the result looks like in the three screenshots below.

The small item has been added to the small box

The medium sized items have been added to the medium sized box

The big box remains empty

Until here the first requirement of assigning each item to the corresponding handling unit is met. However, the target structure (see first screenshot) also requires items to be grouped inside the handling units according to one of their attributes. In this simplified example this attribute is the type of the item. This can easily be achieved by applying operation group by key to the Items inside the handling unit.

Items of different types are grouped inside the second handling unit.

For the second handling unit which actually contains items of two different types, this leads to the creation of two distinct groups of items inside the corresponding handling unit. The fist handling unit in turn will only contain one items group, as it inly contains one distinct type of items. The third Handling unit still remains empty because it doesn’t contain any items to be grouped.

Conclusion

As shown in the picture below, to apply certain structural changes to the source structure of a Message Implementation Guide reduces the main mapping in this case to a simple 1:1 mapping. All this can be done without writing a single line of code. The resulting transformation automatically becomes part of the runtime artifacts and integrates seamlessly with existing integration scenarios respectively integration flows.

Simple 1:1 mapping using the transformed structure of the original source Message Implementation Guide

Please feel free to drop any questions and comments.

For more details, please visit

Integration Advisor – integration game changer

Integration Advisor: Overview of components for building B2B integration content and further reading.

Appendix

Example data used in this example

<DispatchNotificationSource>
    <Header>
        <messageId>0815471142</messageId>
        <senderId>Sender Company</senderId>
        <recipientId>Recipient Number</recipientId>
    </Header>
    <Shipment>
        <HandlingUnit>
            <handlingUnitType>small Box</handlingUnitType>
            <unitId>001</unitId>
            <name>Small Box</name>
        </HandlingUnit>
        <HandlingUnit>
            <handlingUnitType>medium Box</handlingUnitType>
            <unitId>002</unitId>
            <name>Medium Box</name>
        </HandlingUnit>
        <HandlingUnit>
            <handlingUnitType>big Box</handlingUnitType>
            <unitId>003</unitId>
            <name>Big Box</name>
        </HandlingUnit>
        <Item>
            <itemType>small item</itemType>
            <itemId>100</itemId>
            <handlingUnitId>001</handlingUnitId>
            <name>Small Item</name>
        </Item>
        <Item>
            <itemType>small item</itemType>
            <itemId>100</itemId>
            <handlingUnitId>001</handlingUnitId>
            <name>Small Item</name>
        </Item>
        <Item>
            <itemType>light item</itemType>
            <itemId>200</itemId>
            <handlingUnitId>002</handlingUnitId>
            <name>Medium sized Item</name>
        </Item>
        <Item>
            <itemType>heavy item</itemType>
            <itemId>200</itemId>
            <handlingUnitId>002</handlingUnitId>
            <name>Another medium sized Item that is heavy</name>
        </Item>
        <Item>
            <itemType>heavy item</itemType>
            <itemId>200</itemId>
            <handlingUnitId>002</handlingUnitId>
            <name>Yet another medium sized Item that is heavy</name>
        </Item>
    </Shipment>
</DispatchNotificationSource>

Source side structure

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">

    <xs:complexType name="messageHeader">
        <xs:sequence>
            <xs:element name="messageId" type="xs:string" maxOccurs="1"/>
            <xs:element name="senderId" type="xs:string" maxOccurs="1"/>
            <xs:element name="recipientId" type="xs:string" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>

    <xs:complexType name="handlingUnit">
        <xs:sequence>
            <xs:element name="handlingUnitType" type="xs:string" maxOccurs="1"/>
            <xs:element name="unitId" type="xs:string" maxOccurs="1"/>
            <xs:element name="name" type="xs:string" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>

    <xs:complexType name="item">
        <xs:sequence>
            <xs:element name="itemType" type="xs:string" maxOccurs="1"/>
            <xs:element name="itemId" type="xs:string" maxOccurs="1"/>
            <xs:element name="handlingUnitId" type="xs:string" maxOccurs="1"/>
            <xs:element name="name" type="xs:string" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>

    <xs:complexType name="shipment">
        <xs:sequence>
            <xs:element name="HandlingUnit" type="handlingUnit" maxOccurs="unbounded"/>
            <xs:element name="Item" type="item" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>

    <xs:complexType name="dispatchNotification">
        <xs:sequence>
            <xs:element name="Header" type="messageHeader" maxOccurs="1"/>
            <xs:element name="Shipment" type="shipment" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>

    <xs:element name="DispatchNotificationSource" type="dispatchNotification"/>

</xs:schema>

Target side structure

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified">

    <xs:complexType name="messageHeader">
        <xs:sequence>
            <xs:element name="messageId" type="xs:string" maxOccurs="1"/>
            <xs:element name="senderId" type="xs:string" maxOccurs="1"/>
            <xs:element name="recipientId" type="xs:string" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>

    <xs:complexType name="handlingUnit">
        <xs:sequence>
            <xs:element name="handlingUnitType" type="xs:string" maxOccurs="1"/>
            <xs:element name="unitId" type="xs:string" maxOccurs="1"/>
            <xs:element name="name" type="xs:string" maxOccurs="1"/>
            <xs:element name="ItemGroup" type="itemGroup" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>

    <xs:complexType name="itemGroup">
        <xs:sequence>
            <xs:element name="itemTypeInThisGroup" type="xs:string" maxOccurs="1"/>
            <xs:element name="Item" type="item" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>
    
    <xs:complexType name="item">
        <xs:sequence>
            <xs:element name="itemType" type="xs:string" maxOccurs="1"/>
            <xs:element name="itemId" type="xs:string" maxOccurs="1"/>
            <xs:element name="handlingUnitId" type="xs:string" maxOccurs="1"/>
            <xs:element name="name" type="xs:string" maxOccurs="1"/>
        </xs:sequence>
    </xs:complexType>

    <xs:complexType name="shipment">
        <xs:sequence>
            <xs:element name="HandlingUnit" type="handlingUnit" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>

    <xs:complexType name="dispatchNotification">
        <xs:sequence>
            <xs:element name="MessageHeader" type="messageHeader" maxOccurs="1"/>
            <xs:element name="Shipment" type="shipment" maxOccurs="unbounded"/>
        </xs:sequence>
    </xs:complexType>

    <xs:element name="DispatchNotificationTarget" type="dispatchNotification"/>

</xs:schema>