I *heart* Groovy mapping

Choices, choices, choices…

When working on CPI developments, there are a few options for developing mappings, i.e. the transformation of one message format to another. Cloud Integration mapping: Your options explained and compared by Morten Wittrock weighs the different options and their comparison.

On one hand, developers with a PI/PO background may gravitate towards Message Mapping due to familiarity with it.

On the other hand, developers from non-SAP background are normally more familiar with XSLT due to it being an open standard.

Nearly a year after my comment on Morten’s blog post, I am convinced that Groovy indeed is the way to go for anything but the simplest mapping (read: mostly 1-1) in CPI. I invite any and every CPI developer worth his/her salt to view my arguments below and consider adding this skill to their arsenal.

The Case for Groovy

Well, before I prove the case for Groovy, let me prove the case against its “opponents”.

i) Message Mapping (a.k.a Graphical Mapping)

• Relies heavily on a queue and context concept that I have only ever seen a handful of PI developers truly understand
• Does not scale well with complexity (besides the concept above, Groovy is needed anyway if non standard functionality is required)
• Complex logic is difficult to implement especially with the standard functionality. Back in PI days, it is not uncommon to see spiderweb-like mappings like below. It is difficult to decipher, harder to troubleshoot, impossible to enhance without breaking something!

• Cannot be developed offline
• Cannot be tested/simulated offline
• WebUI test simulation cannot test mapping logic that accesses header/property
• Does not scale well with increase of scenarios (how you do ensure logic for a new scenario does not break a previous scenario?)
• I would even dare argue that this is baggage from PI days (an approach at least as old as 15 years) and best not to be used in any new CPI developments

ii) XSLT

• Arguably the steepest learning curve
• Typically limited to XML/Text to XML/Text transformation in the context of system-to-system integration (although it can handle other formats like HTML too)
• Not easy to debug
• To do anything substantial, a good XML editor is required and that typically comes with a price

Ok now, let’s now move to Groovy 🙂

• Pervasive in many areas of CPI (not just mapping) and therefore it is inevitable that a CPI developer will need Groovy irrespective of the mapping approach used
• Capable of any to any transformation
• Tools required for entire lifecycle of development are free or open-source
• Scales well with complexity (due to flexibility of working directly with source code)
• XML transformations are easy to developed (compared to Java) with the use of XmlSlurper for parsing and MarkupBuilder for generation (more details to follow below)
• XmlSlurper has low memory footprint as it is based on SAX
• Can be developed and tested offline
• Mapping logic that accesses header/property can be tested via injection of these values from calling script
• Can be debugged offline (with the help of an IDE like IntelliJ IDEA or Eclipse)
• Transferrable skillset (i.e. to other integration solutions like Mulesoft or Dell Boomi) since payload parsing and generation are platform agnostic
• And last but not least, it scales well with increase of scenarios when unit testing is implemented for each test case. And as a teaser, let me show you the outcome of running 8 tests in just over 3 seconds!

So if you are at least partially convinced or intrigued to find out more, let’s see how this can be achieved practically.

Parsing XML with XmlSlurper

As described in Parse XML easily with Groovy scripts, parsing XML can be achieved easily in Groovy using XmlSlurper. The contents of the fields can be accessed via dot notation. Below shows the comparison between how XML parsing is achieved in Java using DOM versus the simpler Groovy approach.

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder docBuilder = factory.newDocumentBuilder();
Document doc = docBuilder.parse(inputStream);
NodeList nodeList = doc.getElementsByTagName("OrderNumber");
String orderNo = nodeList.item(0).getFirstChild().getTextContent();

def root = new XmlSlurper().parse(reader)
def orderNo = root.Header.OrderNumber

Generating XML with MarkupBuilder

Similarly, MarkupBuilder in Groovy significantly reduces the line of codes for generating XML, as compared to using DOM in Java. The following example shows the comparison for generating a 2-level XML output.

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
DocumentBuilder docBuilder = factory.newDocumentBuilder();
Document doc = docBuilder.newDocument();

Node root = doc.createElement("PurchaseOrder");
doc.appendChild(root);
root.appendChild(doc.createElement("Header"));

Transformer transformer = TransformerFactory.newInstance().newTransformer();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
transformer.transform(new DOMSource(doc), new StreamResult(baos));
String output = new String(baos.toByteArray());

def writer = new StringWriter()
def builder = new MarkupBuilder(writer)
builder.PurchaseOrder {
  'Header'{ }
}
def output = writer.toString()

Sample scenario

Consider a sample integration scenario with the following requirements:-

• Transform payload from source schema to target schema
• Convert date format (dd-MM-yyyy to yyyyMMdd)
• Populate target field DocumentType based on message property (that is configurable via externalised parameter) – Z001 in this example
• Filter only items where IsBatchParent = true

Below are examples of the input payload and expected output payload:-

<Order>
    <Header>
        <OrderNumber>OrderXYZ</OrderNumber>
        <Date>18-02-2019</Date>
    </Header>
    <Item>
        <ItemNumber>10</ItemNumber>
        <MaterialNumber>MatABC</MaterialNumber>
        <Quantity>57</Quantity>
        <IsBatchParent>true</IsBatchParent>
    </Item>
    <Item>
        <ItemNumber>20</ItemNumber>
        <MaterialNumber>MatABC</MaterialNumber>
        <Quantity>57</Quantity>
        <IsBatchParent>false</IsBatchParent>
    </Item>
</Order>

<PurchaseOrder>
    <Header>
        <ID>OrderXYZ</ID>
        <DocumentDate>20190218</DocumentDate>
        <DocumentType>Z001</DocumentType>
    </Header>
    <Item>
        <ProductCode>MatABC</ProductCode>
        <Quantity>57</Quantity>
    </Item>
</PurchaseOrder>

Bringing it all together

With the basics in place, let us see how to bring it all together based on the requirements of the scenario.

def inputDate = LocalDate.parse(input, DateTimeFormatter.ofPattern('dd-MM-yyyy'))
def outputDate = inputDate.format(DateTimeFormatter.ofPattern('yyyyMMdd'))

Map properties = message.getProperties()
def docType = properties.get('DocType')

def validItems = Order.Item.findAll { item -> item.IsBatchParent.text() == 'true' }

With all of this in place, the final source code is listed below:-

import com.sap.gateway.ip.core.customdev.util.Message
import groovy.xml.MarkupBuilder
import java.time.LocalDate
import java.time.format.DateTimeFormatter

Message processData(Message message) {
    // Access message body and properties
    Reader reader = message.getBody(Reader)
    Map properties = message.getProperties()

    // Define XML parser and builder
    def Order = new XmlSlurper().parse(reader)
    def writer = new StringWriter()
    def builder = new MarkupBuilder(writer)

    // Define target payload mapping
    builder.PurchaseOrder {
        'Header' {
            'ID'(Order.Header.OrderNumber)
            def inputDate = LocalDate.parse(Order.Header.Date.text(), DateTimeFormatter.ofPattern('dd-MM-yyyy'))
            'DocumentDate'(inputDate.format(DateTimeFormatter.ofPattern('yyyyMMdd')))
            'DocumentType'(properties.get('DocType'))
        }
        def validItems = Order.Item.findAll { item -> item.IsBatchParent.text() == 'true' }
        validItems.each { item ->
            'Item' {
                'ProductCode'(item.MaterialNumber)
                'Quantity'(item.Quantity)
            }
        }
    }

    // Generate output
    message.setBody(writer.toString())
    return message
}

Note: When a particular node from the source payload is used other than direct assignment to the target node, it needs to be transformed to a text field by adding .text() method at the end of the node.

Setting up Unit Tests

Next, to make it worth the effort, we will set up the above scenario as a test case for unit testing. Once again I will use the Spock-based approach, such that once we have a successful test case, there will be a baseline to ensure any future changes to the Groovy script would not break an existing scenario.

The development will be maintained as an IntelliJ project based on the below shown directory structure as described in my previous post, Pimp My Groovy – boosting CPI Groovy developments with IntelliJ IDEA. The script under test will be OrderToPurchaseOrder.groovy, and the Spock specification will be OrderToPurchaseOrderSpec.groovy. The input XML payload and expected XML output payload are maintained as resources in the project.

Following is the source code for the Spock specification, with the single test case. The input file is stored into the message body together with the parameter for DocType. The Groovy mapping script is then executed and the results compared with the provided output file.

package src.test.groovy

import com.sap.gateway.ip.core.customdev.processor.MessageImpl
import com.sap.gateway.ip.core.customdev.util.Message
import org.apache.camel.CamelContext
import org.apache.camel.Exchange
import org.apache.camel.impl.DefaultCamelContext
import org.apache.camel.impl.DefaultExchange
import spock.lang.Shared
import spock.lang.Specification

class OrderToPurchaseOrderSpec extends Specification {

    Message msg
    Exchange exchange
    @Shared script

    def setup() {
        // Load Groovy Script
        GroovyShell shell = new GroovyShell()
        script = shell.parse(new File('src/main/resources/script/OrderToPurchaseOrder.groovy'))

        CamelContext context = new DefaultCamelContext()
        exchange = new DefaultExchange(context)
        msg = new MessageImpl(exchange)
    }

    def 'Purchase Order Mapping for Document Type Z001'() {
        given:
        //--------------------------------------------------------------
        // Initialize message with body, header and property
        def body = new File('src/test/resources/Order.xml')
        msg.setProperty('DocType', 'Z001')
        //--------------------------------------------------------------

        exchange.getIn().setBody(body)
        msg.setBody(exchange.getIn().getBody())

        when:
        // Execute script
        script.processData(msg)

        then:
        msg.getBody() == new File('src/test/resources/OrderOutput.xml').text
    }
}

Finally, once the Spock test is executed in IntelliJ, the below results are shown in the console.

Conclusion

Unashamedly, I love Groovy and the boundless options it provides in the context of CPI development. Specifically in the area of mapping, Groovy mappings are my first preference given the ease of implementation, supported by the right toolset. It further allows us to safeguard our development with the easy incorporation of unit testing capability.

If you are contemplating the best approach for performing mapping in CPI, I strongly recommend Groovy mapping.