I *heart* Groovy mapping

engswee · ‎06-07-2019

Choices, choices, choices...

When working on CPI developments, there are a few options for developing mappings, i.e. the transformation of one message format to another. Cloud Integration mapping: Your options explained and compared by 7a519509aed84a2c9e6f627841825b5a weighs the different options and their comparison.

On one hand, developers with a PI/PO background may gravitate towards Message Mapping due to familiarity with it.

On the other hand, developers from non-SAP background are normally more familiar with XSLT due to it being an open standard.

Nearly a year after my comment on Morten's blog post, I am convinced that Groovy indeed is the way to go for anything but the simplest mapping (read: mostly 1-1) in CPI. I invite any and every CPI developer worth his/her salt to view my arguments below and consider adding this skill to their arsenal.

The Case for Groovy

Well, before I prove the case for Groovy, let me prove the case against its "opponents".

i) Message Mapping (a.k.a Graphical Mapping)

Relies heavily on a queue and context concept that I have only ever seen a handful of PI developers truly understand

Does not scale well with complexity (besides the concept above, Groovy is needed anyway if non standard functionality is required)

Complex logic is difficult to implement especially with the standard functionality. Back in PI days, it is not uncommon to see spiderweb-like mappings like below. It is difficult to decipher, harder to troubleshoot, impossible to enhance without breaking something!

Cannot be developed offline

Cannot be tested/simulated offline

WebUI test simulation cannot test mapping logic that accesses header/property

Does not scale well with increase of scenarios (how you do ensure logic for a new scenario does not break a previous scenario?)

I would even dare argue that this is baggage from PI days (an approach at least as old as 15 years) and best not to be used in any new CPI developments

ii) XSLT

Arguably the steepest learning curve

Typically limited to XML/Text to XML/Text transformation in the context of system-to-system integration (although it can handle other formats like HTML too)

Not easy to debug

To do anything substantial, a good XML editor is required and that typically comes with a price

Ok now, let's now move to Groovy 🙂

Pervasive in many areas of CPI (not just mapping) and therefore it is inevitable that a CPI developer will need Groovy irrespective of the mapping approach used

Capable of any to any transformation

Tools required for entire lifecycle of development are free or open-source

Scales well with complexity (due to flexibility of working directly with source code)

XML transformations are easy to developed (compared to Java) with the use of XmlSlurper for parsing and MarkupBuilder for generation (more details to follow below)

XmlSlurper has low memory footprint as it is based on SAX

Can be developed and tested offline

Mapping logic that accesses header/property can be tested via injection of these values from calling script

Can be debugged offline (with the help of an IDE like IntelliJ IDEA or Eclipse)

Transferrable skillset (i.e. to other integration solutions like Mulesoft or Dell Boomi) since payload parsing and generation are platform agnostic

And last but not least, it scales well with increase of scenarios when unit testing is implemented for each test case. And as a teaser, let me show you the outcome of running 8 tests in just over 3 seconds!

So if you are at least partially convinced or intrigued to find out more, let's see how this can be achieved practically.

Parsing XML with XmlSlurper

As described in Parse XML easily with Groovy scripts, parsing XML can be achieved easily in Groovy using XmlSlurper. The contents of the fields can be accessed via dot notation. Below shows the comparison between how XML parsing is achieved in Java using DOM versus the simpler Groovy approach.

Java

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();

DocumentBuilder docBuilder = factory.newDocumentBuilder();

Document doc = docBuilder.parse(inputStream);

NodeList nodeList = doc.getElementsByTagName("OrderNumber");

String orderNo = nodeList.item(0).getFirstChild().getTextContent();

Groovy

def root = new XmlSlurper().parse(reader)

def orderNo = root.Header.OrderNumber

Generating XML with MarkupBuilder

Similarly, MarkupBuilder in Groovy significantly reduces the line of codes for generating XML, as compared to using DOM in Java. The following example shows the comparison for generating a 2-level XML output.

Java

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();

DocumentBuilder docBuilder = factory.newDocumentBuilder();

Document doc = docBuilder.newDocument();



Node root = doc.createElement("PurchaseOrder");

doc.appendChild(root);

root.appendChild(doc.createElement("Header"));



Transformer transformer = TransformerFactory.newInstance().newTransformer();

ByteArrayOutputStream baos = new ByteArrayOutputStream();

transformer.transform(new DOMSource(doc), new StreamResult(baos));

String output = new String(baos.toByteArray());

Groovy

def writer = new StringWriter()

def builder = new MarkupBuilder(writer)

builder.PurchaseOrder {

  'Header'{ }

}

def output = writer.toString()

Sample scenario

Consider a sample integration scenario with the following requirements:-

Transform payload from source schema to target schema

Convert date format (dd-MM-yyyy to yyyyMMdd)

Populate target field DocumentType based on message property (that is configurable via externalised parameter) - Z001 in this example

Filter only items where IsBatchParent = true

Below are examples of the input payload and expected output payload:-

Input

<Order>

    <Header>

        <OrderNumber>OrderXYZ</OrderNumber>

        <Date>18-02-2019</Date>

    </Header>

    <Item>

        <ItemNumber>10</ItemNumber>

        <MaterialNumber>MatABC</MaterialNumber>

        <Quantity>57</Quantity>

        <IsBatchParent>true</IsBatchParent>

    </Item>

    <Item>

        <ItemNumber>20</ItemNumber>

        <MaterialNumber>MatABC</MaterialNumber>

        <Quantity>57</Quantity>

        <IsBatchParent>false</IsBatchParent>

    </Item>

</Order>

Output

<PurchaseOrder>

    <Header>

        <ID>OrderXYZ</ID>

        <DocumentDate>20190218</DocumentDate>

        <DocumentType>Z001</DocumentType>

    </Header>

    <Item>

        <ProductCode>MatABC</ProductCode>

        <Quantity>57</Quantity>

    </Item>

</PurchaseOrder>

Bringing it all together

With the basics in place, let us see how to bring it all together based on the requirements of the scenario.

Requirement	Approach
Transform payload from source schema to target schema	Utilise XmlSlurper to parse the input XML, and MarkupBuilder to generate the output XML
Convert date format (dd-MM-yyyy to yyyyMMdd)	Use Java 8's LocalDate to parse and format the date `def inputDate = LocalDate.parse(input, DateTimeFormatter.ofPattern('dd-MM-yyyy')) def outputDate = inputDate.format(DateTimeFormatter.ofPattern('yyyyMMdd'))`
Populate target field DocumentType based on message property	Access via the Message property `Map properties = message.getProperties() def docType = properties.get('DocType')`
Filter only items where IsBatchParent = true	Use GPathResult's findAll method by providing a Closure with the appropriate filter criteria `def validItems = Order.Item.findAll { item -> item.IsBatchParent.text() == 'true' }`

With all of this in place, the final source code is listed below:-

import com.sap.gateway.ip.core.customdev.util.Message

import groovy.xml.MarkupBuilder

import java.time.LocalDate

import java.time.format.DateTimeFormatter



Message processData(Message message) {

    // Access message body and properties

    Reader reader = message.getBody(Reader)

    Map properties = message.getProperties()



    // Define XML parser and builder

    def Order = new XmlSlurper().parse(reader)

    def writer = new StringWriter()

    def builder = new MarkupBuilder(writer)



    // Define target payload mapping

    builder.PurchaseOrder {

        'Header' {

            'ID'(Order.Header.OrderNumber)

            def inputDate = LocalDate.parse(Order.Header.Date.text(), DateTimeFormatter.ofPattern('dd-MM-yyyy'))

            'DocumentDate'(inputDate.format(DateTimeFormatter.ofPattern('yyyyMMdd')))

            'DocumentType'(properties.get('DocType'))

        }

        def validItems = Order.Item.findAll { item -> item.IsBatchParent.text() == 'true' }

        validItems.each { item ->

            'Item' {

                'ProductCode'(item.MaterialNumber)

                'Quantity'(item.Quantity)

            }

        }

    }



    // Generate output

    message.setBody(writer.toString())

    return message

}

Note: When a particular node from the source payload is used other than direct assignment to the target node, it needs to be transformed to a text field by adding .text() method at the end of the node.

Setting up Unit Tests

Next, to make it worth the effort, we will set up the above scenario as a test case for unit testing. Once again I will use the Spock-based approach, such that once we have a successful test case, there will be a baseline to ensure any future changes to the Groovy script would not break an existing scenario.

The development will be maintained as an IntelliJ project based on the below shown directory structure as described in my previous post, Pimp My Groovy – boosting CPI Groovy developments with IntelliJ IDEA. The script under test will be OrderToPurchaseOrder.groovy, and the Spock specification will be OrderToPurchaseOrderSpec.groovy. The input XML payload and expected XML output payload are maintained as resources in the project.

Following is the source code for the Spock specification, with the single test case. The input file is stored into the message body together with the parameter for DocType. The Groovy mapping script is then executed and the results compared with the provided output file.

package src.test.groovy



import com.sap.gateway.ip.core.customdev.processor.MessageImpl

import com.sap.gateway.ip.core.customdev.util.Message

import org.apache.camel.CamelContext

import org.apache.camel.Exchange

import org.apache.camel.impl.DefaultCamelContext

import org.apache.camel.impl.DefaultExchange

import spock.lang.Shared

import spock.lang.Specification



class OrderToPurchaseOrderSpec extends Specification {



    Message msg

    Exchange exchange

    @Shared script



    def setup() {

        // Load Groovy Script

        GroovyShell shell = new GroovyShell()

        script = shell.parse(new File('src/main/resources/script/OrderToPurchaseOrder.groovy'))



        CamelContext context = new DefaultCamelContext()

        exchange = new DefaultExchange(context)

        msg = new MessageImpl(exchange)

    }



    def 'Purchase Order Mapping for Document Type Z001'() {

        given:

        //--------------------------------------------------------------

        // Initialize message with body, header and property

        def body = new File('src/test/resources/Order.xml')

        msg.setProperty('DocType', 'Z001')

        //--------------------------------------------------------------



        exchange.getIn().setBody(body)

        msg.setBody(exchange.getIn().getBody())



        when:

        // Execute script

        script.processData(msg)



        then:

        msg.getBody() == new File('src/test/resources/OrderOutput.xml').text

    }

}

Finally, once the Spock test is executed in IntelliJ, the below results are shown in the console.

Conclusion

Unashamedly, I love Groovy and the boundless options it provides in the context of CPI development. Specifically in the area of mapping, Groovy mappings are my first preference given the ease of implementation, supported by the right toolset. It further allows us to safeguard our development with the easy incorporation of unit testing capability.

If you are contemplating the best approach for performing mapping in CPI, I strongly recommend Groovy mapping.