Compare 2 payloads to identify changes using Groovy Script in SAP CI

Introduction:
This blog will help you solve the requirement to compare different payloads and identify changes using Groovy script in SAP CPI.

Many times, source system doesn’t send the data with differentiation of new/updated record type and target system only expects the records which are either new or got updated in source. In this case middleware has to identify the record type using custom logic.

This blog explains about the Groovy scripts used to achieve this requirement. In the below scenario, I have taken 2 sample XML payloads to explain it in Groovy script in SAP CI.

Scenario:

Step 1- Create an integration flow with required flow steps.

Here I have added 1st payload in the 1st content modifier body and 2nd payload in the 3rd content modifier body. 1st payload is the primary one which will be compared with 2nd. In the actual scenario these payloads will be coming from external source via request-reply flow step.

1st Payload:

2nd Payload:

Step 2 – Add below code in the 1st groovy script.

def Message compareUniqueIdFromPayloads(Message message)
 {
    //Body
    def body = message.getBody(java.lang.String) as String;      
    def payload2 = new XmlParser().parse(message.getBody(java.io.Reader));
    def properties = message.getProperties();
    def system1Payload = message.getProperties().get("System1Payload")
    def payload1 = new XmlParser().parseText(system1Payload)
    
    def payload1Node = "";
    def payload2Node = "";
    def nodeMatching = "false";
    
    for(int i=0;i<payload1.Category.size();i++)
    {
       payload1Node = payload1.Category[i].CategoryCode.text();
       nodeMatching = "false";
      
        for(int j=0;j<payload2.row.size();j++)
        {
           payload2Node = payload2.row[j].WHCode.text();
           if(payload1Node == payload2Node)
           {
               nodeMatching = "true";
           }
           else
           {
             nodeMatching = "false";  
           }
         
        }
        if(nodeMatching == "true")
        {
           //Add new node "RecordType" with "ExistingRecord" in the payload1
           def newNode = "<RecordType>" + "ExistingRecord" + "</RecordType>";
           def newNodeValue = new XmlSlurper().parseText(newNode);
           payload1.Category[i].appendNode("RecordType", [:], "ExistingRecord");
        }
        else
        {
           //Add new node "RecordType" with "NewRecord" in the payload1
           def newNode = "<RecordType>" + "NewRecord" + "</RecordType>";
           def newNodeValue = new XmlSlurper().parseText(newNode);
           payload1.Category[i].appendNode("RecordType", [:], "NewRecord");
           
        }  
    } 
    //Final message body creation
    def result = (String)XmlUtil.serialize(payload1)  
    result = result.replaceAll("\n *?\n", "\n").trim()
    message.setBody(result) 
    return message;
}

Output payload with new created node <RecordType> in each record which will identify whether this record is newly created or already existed in the source system.

Step 3 – Add below code in the 2nd groovy script.

def Message compareRemainingFields(Message message)
 {
    //Body
    def body = message.getBody(java.lang.String) as String;
    def payload1 = new XmlSlurper().parseText(body);

    def properties = message.getProperties();
    def system2Payload = properties.get("System2Payload");
    def payload2 = new XmlSlurper().parseText(system2Payload);

    def categoryCode = "";
    def categoryName = "";
    def categoryCurrency = "";
    def whCode = "";
    def whName = "";
    def whCurrency = "";
    def nodeMatching = "false";
    def createNewRecord = "false";
    def recordtype = "";
    //Loop through payload1
    for(int i=0; i<payload1.Category.size();i++) 
    {
        categoryCode = payload1.Category[i].CategoryCode.text();
        categoryName = payload1.Category[i].CategoryName.text();
        categoryCurrency = payload1.Category[i].CategoryCurrency.text();
        recordtype = payload1.Category[i].RecordType.text();
        nodeMatching = "false";

        if(recordtype == "ExistingRecord") 
        {
            //Loop through payload2
            for(int j=0; j<payload2.row.size();j++) 
            {
                whCode = payload2.row[j].WHCode.text();
                //Compare unique id field between 2 payloads
                if(whCode==categoryCode) 
                {
                    whName = payload2.row[j].WHName.text();
                    whCurrency = payload2.row[j].WHCurrency.text();
                    if((categoryName == whName) && (categoryCurrency == whCurrency))
                    {
                        //Add new node "RecordAction" with "IgnoreRecord" in the payload1
                        def newNode = "<RecordAction>" + "IgnoreRecord" + "</RecordAction>";
                        def newNodeValue = new XmlSlurper().parseText(newNode);
                        payload1.Category[i].appendNode(newNodeValue);


                    } 
                    else
                    {
                        //Add new node "RecordAction" with "UpdateRecord" in the payload1
                        def newNode = "<RecordAction>" + "UpdateRecord" + "</RecordAction>";
                        def newNodeValue = new XmlSlurper().parseText(newNode);
                        payload1.Category[i].appendNode(newNodeValue);
                    }
                    break;
                }
            }
        }
        else 
        {
            //Add new node "RecordAction" with "CreateRecord" in the payload1
            def newNode = "<RecordAction>" + "CreateRecord" + "</RecordAction>";
            def newNodeValue = new XmlSlurper().parseText(newNode);
            payload1.Category[i].appendNode(newNodeValue);
        }

    }
    //Final message body creation
    def result = (String)XmlUtil.serialize(payload1)  
    result = result.replaceAll("\n *?\n", "\n").trim()
    message.setBody(result) 
    return message;
}

Output payload with new created node <RecordAction> in each record which will identify whether we have to create/update the record in the target system or to ignore it.

Step 4 – Add below code in the 3rd groovy script. 

This code will remove records which are neither new nor changed in the source system.

def Message removeUnwantedNodes(Message message)
{
    def body = message.getBody(java.lang.String) as String; 
    def properties = message.getProperties();
    
    def pfile = new XmlParser().parseText(body);
    //Loop through nodes to identify unwanted record and delete it from final payload
    pfile.Category.each{categoryNode ->
            if(categoryNode.RecordAction.text() == "IgnoreRecord")
                {
                    pfile.remove(categoryNode);
                }
    }
    //Final message body creation
    message.setBody(XmlUtil.serialize(pfile));
    return message;
}

Step 5 – Final Payload

Final payload will only have new and updated records. Now you can do the mapping and implement further business logics based on RecordType/RecordAction node.

Conclusion:

So, to conclude, this blog post gives you an idea on how to compare different payloads and modify it by adding new nodes and deleting the unwanted ones.

If you have any queries, please feel free to ask your question in comments. I would request everyone to provide your feedback and like if this blog post finds helpful for you.