SAP CPI - Parsing Mechanisms - GroovyScript

rhviana · ‎02-13-2020

Hello Folks,

Returning to the cloud topics, I would like share with you my experience with Apache Groovy Script parsing mechanism for XML file using available classes to just replace the structure or build from the starch the output XML file.

I developed 7 years ago of DOM Parsing via java mapping - SAP PI/PO:

Java Mapping

The proposal of this blog it's give you knowledge and present the CLASSES of parsing XML files available in Groovy Script to SAP CPI, not the integration aspects.

You will be able to replicate and others with the samples of codes, enjoy, this is compilation of this main source of information - Reference of some texts below:

Processing XML with Groovy

Why not JSON format ?

I developed this in XML in 2013 the year that JSON was released and I just would like to replicate the similar scenario off course that it's somehow similar in case of JSON input.

Agenda:

First steps - Install Groovy Plugin in NWDS

On-line compiler

Differences of XMLParser vs XMLSluper?

Key difference of classes?

How to make decision?

Gpath Result

MarkupBuilder - StreamingMarkupBuilder

Dom Category

My decision witch class to use?

Integration Diagram view

Why Java mapping and now Groovy ?

The mapping requirement for this development

Integration Flow - CPI with XmlSluper and StreamingMarkupBuilder:
- CPI - XML Supler
- Why XSLT after?
- CPI - XML StreamingMarkupBuilder
- Postman execution
- Online Compiler perspective
- Code using XMLSluper
- Code using StreamingMarkupBuilder

Comparison of line code of Java and Groovy

Recommendations

Thanks and credits

1 ) First Steps - Install Groovy Plugin in NWDS:

I will explain you in details how to install the groovy script plugging in the SAP Netweaver Developer Studio to be easier to build and test, instead of directly coding and compilation in the SAP Cloud Platform Integration - CPI.

Download the SAP Netweaver Developer Studio from SAP Marketplace 7.5 sp16.

Version Oxygen:

Github with all plugins and versions of the groovy script:

Many releases can be found in Github - GroovyScript, in particular, you may want to try the "snapshot builds" instead of the release builds.

If you are using the Netweaver Developer Studio you must set up for Oxygen:

Oxygen - SAP Netweaver Developer Studio 7.5 sp16

Go to Help > Install New Software

Install New Software > Add

Add Repository:

Name: Groovy Development Eclipse (Suggestion, you can add anything)

Location: http://dist.springsource.org/snapshot/GRECLIPSE/e4.7 - Select all packages

Next Steps:

Click the Next button twice

Select "I accept the terms and conditions"

Click Finish button

Once updates have finished you must restart the Netweaver Developer Studio, click Restart Now.

You can now do File > New > Project > Groovy > Groovy Project

Check if the installation is with success:

Or here - Help > Eclipse Marketplace > Installed

Or here - Help > About Eclipse

Creating the groovy project in SAP Netweaver Developer Studio.

New Project > Groovy Others > Groovy

Now you can develop in your local SAP Netweaver developer studio and play around reading and writing local files before upload in the SAP CPI.

2 ) Online compiler:

Also you can use these very nice web console for Groovy if you don't want to keep your projects in local machine.

Groovy Web Console

Coding groud

3 ) Differences of XMLParser vs XMLSluper?

The parses of XmlParser and XmlSlurper (similar to DOM or SAX in java) are pretty similar. We can more or less achieve the same results with both. However, some differences between them can tilt the scales towards one or the other.

First of all, XmlParser always parses the whole document into the DOM-ish structure. Because of that, we can simultaneously read from and write into it. We can’t do the same with XmlSlurper as it evaluates paths more lazily. As a result, XmlParser can consume more memory.

On the other hand, XmlSlurper uses more straightforward definitions, making it simpler to work with. We also need to remember that any structural changes made to XML using XmlSlurper require reinitialization, which can have an unacceptable performance hit in case of making many changes one after another.

Let’s see the similarities between XmlParser and XmlSlurper first:

Both can update/transform the XML

4 ) Key difference of classes:

XmlSlurper:
1. Evaluates the structure lazily. So if you update the XML you’ll have to evaluate the whole tree again.
2. Returns GPath Result instances when parsing XML
3. Consumes Lesser Memory Than XML Parser.

XmlParser:
1. Returns Almost All Nodes objects when parsing XM
2. Is Faster Then XML Slurper.
3. Can Parse And Update The XML Simultaneously.

5 ) How to make decision?

The decision of which tool to use should be made with care and depends entirely on the use case.

If you want to transform an existing document to another then XmlSlurper will be the choice

If you want to update and read at the same time then XmlParser is the choice.

If you just have to read a few nodes XmlSlurper should be your choice, since it will not have to create a complete structure in memory"

In general, both classes perform similar ways. Even the way of using GPath expressions with them are the same (both use breadthFirst() and depthFirst() expressions or (XmlSlurper .'**'.find { it.name() == 'Node' })

6 ) Gpath Result:

GPath is a path expression language integrated into Groovy which allows parts of nested structured data to be identified. In this sense, it has similar aims and scope as XPath does for XML. The two main places where you use GPath expressions is when dealing with nested POJOs or when dealing with XML

7 ) MarkupBuilder - StreamingMarkupBuilder

Apart from reading and manipulating the XML tree, Groovy also provides tooling to create an XML document from scratch. Let’s now create a document consisting of the first two articles from our first example using groovy.xml.MarkupBuilder:

8 ) Dom Category

There is another way of parsing XML documents with Groovy with the use of groovy.xml.dom.DOMCategory which is a category class which adds GPath style operations to Java’s DOM classes.

Java has in-built support for DOM processing of XML using classes representing the various parts of XML documents, e.g. Document, Element, NodeList, Attr, etc. For more information about these classes, refer to the respective JavaDocs.

9 ) My decision witch class to use?

I will present you only the - XmlSlurper and StreamingMarkupBuilder

10 ) Integration Diagram view

Proof of concept with Postman simulating the backend system via HTTPS call, SAP CPI platform receiving the message steps below:

HTTPS Call - POST

2 Groovy script to parse the XML using XmlSlurper(),StreamingMarkupBuilder() and Logging()

XSLT - Just to preserve the format of <![CDATA<Business Data>]> - Only for XmlSlurper and XmlParser

Groovy Script to log the result

End process.

11) Why Java mapping and now Groovy ?

Because the SAP PI and SAP CPI graphical mapping is not able to parse this logic only with the standard functions, off course you could use UDF for that, but in 2013 I accept the challenge without any experience in JAVA to develop one of my first java mapping solution.

I just decide replicate the same now in SAP CPI using groovy, to learn and share the knowledge with you in the level of anyone can replicate and understand why I did like that the solution.

12 ) The mapping requirement for this development

The XML input - MT_GraosSAP_Resp:

BlocoFI - related with SAP FI/CO

BlocoMM - related with SAP MM

BlocoNFe related with Electronic Invoices Brazil

The XML output - ReceiveMovimientoAsientosNotaFiscal

AsientoResponse - related with BlocoFI

MovimientoResponse - related with BlocoMM

NotaFiscalResponse - BlocoNFe plus custom details from the tag <StringXMLNFe>

Picture detail of response Input and response Output prespective:

Is it clear, right ?

If you don't understand the requirement yet, please check more details below:

Mapping 1:1 for this structures, you can use Graphical mapping for that before your groovy to be more easy to parsing

BlocoFI → AssientoReponse

BlocoMM → MovimientoResponse

Important information:

There is a bug in the graphical mapping if you call from external software that it's not able to parsing the CDATA String ennabled inside the main XML

I really don't have any idea, I could not find fast any note or something about that and because of that, I decided to develop the XSLT transformer after the groovy to preserve the <CDATA>

Graphical mapping BUG when I tried to just simulate the test internally.

Original XML imported in SAP CPI for test:

Mapping requirement in details for output <NotaFiscalResponse> from the XML string NF-e enabled in <StringXMLNFe> input response message:

The requirement was build the <NotaFiscalResponse> with 1:1 mapping for some values but the <RespustaSefaz> must be created with the values of XML NF-e CDATA string as you can see below

I believe it's clear, correct ? 🙂

Now let's go inside SAP CPI and how I did that .

13) Integration Flow - CPI with XmlSluper and StreamingMarkupBuilder:

Integration Flow details:

Sender

Groovy XmlSlurper Class or Groovy StreamingMarkupBuilder class

XSLT Mapping only if you use XMLSluper to preserd the <![CDATA]>

End Message

Why not graphical mapping Viana ?

Please check the BUG above so I decide to parsing all input message via groovy.

CPI - XML Supler

Why XSLT after?

XSLT to preserve CDATA XML enabled XML inside the <StringXMLNFe> tag.

This only needs in case of you decide parsing using XmlSlurper and XmlParser both classes are not able to CDATA as you can see in point 11.5 Online compiler perspective.

XSLT Mapping program detail:

<?xml version="1.0" encoding="UTF-8"?>



<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0">



    <! --cdata-section-elements maintain the string inside <![CDATA<Business Data>]> -->



   <xsl:output method="xml" encoding="utf-8" indent="yes" cdata-section-elements="XMLNFe" />



   <! -- The node()|@* is matching all child nodes (node() is all text,element,processing instructions,comments)



   and attributes (@*) of the current context.  -->



   <xsl:template match="@*|node()">



      <xsl:copy>



         <xsl:apply-templates select="@*|node()" />



      </xsl:copy>



   </xsl:template>



</xsl:stylesheet>

Groovy's XmlSlurper and XmlParser don't preserve <![CDATA[<XML>]]> as I mentioned before.

CPI - StreamingMarkupBuilder:

Postman execution:

Request message:

Postman tool under the SAP CPI endpoint with content-type application/xml, after you deploy the scenario with sender HTTP you copy the endpoint.

Response message - HTTPS calls is synchronous default behavior in SAP CPI ( you don't need use the Request-Replay integration pattern)

Online Compiler perspective:

Code use XMLSluper

The code will be shared with link of On-Line Compiler with all details, if you find something wrong let me know.

Using XMLSluper you must use graphical mapping (CPI BUG) or XSLT before, because this code it's the result message with REPLACE function.

XML Sluper

Code use StreamingMarkupBuilder

The code will be shared with link of On-Line Compiler with all details, if you find something wrong let me know.

StreamingMarkupBuilder

14 ) Comparison of line code of Java and Groovy

Expected result as a similar image from the 7 years old DOM Parsing blog:

Result SAP PI - Javamapping DOM Parsing:

Result in SAP CPI using:

Send HTTPS call on CPI per default configuration it's synchronous, so the image below it's the replay after logging message, look that the parsing Groovy Parsing XmlSlurper works fine and insert the node RespustaSefaz after <ok>

Postman CALL

Online compiler

As you can see in the on-line compiler I could not use the XSLT to perserve the <![CDATA[xml]]>

Sample: <XMLNFe><?xml version="1.0" encoding="UTF-8"?>

Java (DOM parsing) - 40 lines just for some part of the code

Java DOM Paring part of code - 40 lines:

________________________________________

/*------------------------------------------------------------------------------

* Creating elements

*------------------------------------------------------------------------------*/

respostaSefaz = doc.createElement("ns1:RespuestaSefaz");

autorizadaPorSefaz = doc.createElement("ns1:AutorizadaPorSefaz");

claveAcceso = doc.createElement("ns1:ClaveAcceso");

codigo = doc.createElement("ns1:Codigo");

codigoAmbiente = doc.createElement("ns1:CodigoAmbiente");

codigoEstado = doc.createElement("ns1:CodigoEstado");

digestValue = doc.createElement("ns1:DigestValue");

digitoVerificador=doc.createElement("ns1:DigitoVerificador");

fechaRecibido = doc.createElement("ns1:FechaRecibido");

motivo = doc.createElement("ns1:Motivo");

nroNotaFiscal = doc.createElement("ns1:NroNotaFiscal");

nroSerieNotaFiscal = doc.createElement("ns1:NroSerieNotaFiscal");

nroProtocolo = doc.createElement("ns1:NumeroProtocolo");

/*------------------------------------------------------------------------------

* Inserting the values into new tags from values extracts

*------------------------------------------------------------------------------*/

autorizadaPorSefaz.appendChild(doc.createTextNode(cStat.replace(cStat, "true")));

claveAcceso.appendChild(doc.createTextNode(chNFe));

codigo.appendChild(doc.createTextNode(cNF));

codigoAmbiente.appendChild(doc.createTextNode(tpAmb));

codigoEstado.appendChild(doc.createTextNode(cStat));

digestValue.appendChild(doc.createTextNode(digestVal));

fechaRecibido.appendChild(doc.createTextNode(dhRecbto));

motivo.appendChild(doc.createTextNode(xMotivo));

nroNotaFiscal.appendChild(doc.createTextNode(nNF));

nroSerieNotaFiscal.appendChild(doc.createTextNode(serie));

nroProtocolo.appendChild(doc.createTextNode(nProt));

/*---------------------------------------------------------------------

* Creating the <ns1:RespuestaSefaz> and inserting child tags

*---------------------------------------------------------------------*/

respostaSefaz.appendChild(autorizadaPorSefaz);

respostaSefaz.appendChild(nroNotaFiscal);

respostaSefaz.appendChild(nroSerieNotaFiscal);

respostaSefaz.appendChild(claveAcceso);

respostaSefaz.appendChild(codigoAmbiente);

respostaSefaz.appendChild(codigoEstado);

respostaSefaz.appendChild(digestValue);

respostaSefaz.appendChild(fechaRecibido);

respostaSefaz.appendChild(motivo);

respostaSefaz.appendChild(nroNotaFiscal);

respostaSefaz.appendChild(nroSerieNotaFiscal);

rootNFResponse.insertBefore(respostaSefaz, tagXmlNFe);

absTraceLog.addDebugMessage(" O elemento <"+respostaSefaz.getNodeName()+"> foi inserido acima da tag <"+ tagXmlNFe.getNodeName() + ">"

+ "que fica na estrutura <"+ rootNFResponse.getNodeName() + ">");

source = new DOMSource(doc);

result = new StreamResult(tOut.getOutputPayload().getOutputStream());

tf.transform(source, result);

}

________________________________________

Groovy do the same of this JAVA DOM parsing in only 24 lines, 50% LESS.

15 ) Recommendations:

If you are interested to learn more I recommend you to buy SAP press book of engswee.yeoh and vadim.klimov - Groovy - check the blog below:

- Groovy SAP Press

16 ) Thanks and credits:

I would like to thank you…

Let's referral all professional that post something related to Groovy Script that was relevant for this PoC - in SAP CPI Platform:

I hope that you like my innovative way to present knowledge and good context of blogs and get a real knowledge how to use Groovy to parsing complex contexts changes in XML, build or just replace structures as a output result message.

Kind Regards,

Viana.

Why not JSON format ?

Agenda:

First steps - Install Groovy Plugin in NWDS

On-line compiler

Differences of XMLParser vs XMLSluper?

Key difference of classes?

How to make decision?

Gpath Result

MarkupBuilder - StreamingMarkupBuilder

Dom Category

My decision witch class to use?

Integration Diagram view

Why Java mapping and now Groovy ?

The mapping requirement for this development

Integration Flow - CPI with XmlSluper and StreamingMarkupBuilder:

CPI - XML Supler

Why XSLT after?

CPI - XML StreamingMarkupBuilder

Postman execution

Online Compiler perspective

Code using XMLSluper

Code using StreamingMarkupBuilder

Comparison of line code of Java and Groovy

Recommendations

Thanks and credits