This blog is (once more) about Integration Gateway in SAP Mobile Platform 3.0.
It is meant for those of you who have already created OData services based on REST data source through Integration Gateway in SAP Mobile Platform 3.0 (SMP).
If you’ve implemented the script as described in my previous tutorial and if you’ve wondered if there isn’t a different way to convert the payload than just doing string operations… (only) then you should have a look at the following blog.
In the bottom of my heart… I was wondering if string operations are the only way to convert a payload from one format to another one.
The REST services that I had tried, are providing nicely structured xml payload in their response body.
So why not parse it with an XML parser?
In this blog, I’m describing how I’ve written a prototype for manipulating the payload of a REST service with an XML parser (instead of string operations).
This is not an official guide.
It might not meet all your requirements – so please do let me know your concerns or suggestions for improvements.
This tutorial is based on SMP SP05.
Prerequisites
- My previous tutorial where we’ve created an OData service based on a REST data source
- SAP Mobile Platform 3.0 SP 05
- Eclipse Kepler with SAP Mobile Platform Tools installed
Preparation
SAP Mobile Platform
As described in one of my previous tutorials you need to create a destination that points to the host of the REST service that is used in this tutorial:
with "no Authentication".
Eclipse
Create OData Implementation Project.
Create OData model or better: import the model which we used in the previous tutorial
Create binding to our example REST service: /street/streetsearch?lat=41.89&lng=-87.64
Create Custom Code script for Groovy.
The following tutorial shows how to implement the script.
Implementing the data structure conversion using XML parser
Overview
We will be implementing only the method processResponseData()
The steps are:
- Get the payload string
- Parse the xml / convert to DOM
- Refactor the DOM
- Transform the DOM to String
- Set the converted payload string
1. Step: Get the payload
This is the same step a usual:
String payload = message.getBody().toString();
2. Step: Parse the XML
In this tutorial, we’re using w3c.dom - API
But before we can parse the response body, we have to transform it to InputSource.
This is the code:
def Document parsePayload(String payload) {
InputStream inputStream = new ByteArrayInputStream(payload.getBytes(StandardCharsets.UTF_8));
InputSource inputSource = new InputSource(inputStream);
DocumentBuilder parser;
try {
parser = DocumentBuilderFactory.newInstance().newDocumentBuilder();
// now parse
return parser.parse(inputSource);
} catch (ParserConfigurationException e) {
log.logErrors(LogMessage.TechnicalError, "Error: failed to create parser");
return null;
} catch (SAXException e) {
log.logErrors(LogMessage.TechnicalError, "Exception ocurred while parsing the response body");
return null;
}
}
3. Step: Refactor the Document
Quick recap: what do we have to do?
This is the response body that we get from the REST service:
And this is the structure that is expected by Integration Gateway:
<EntitySetName>
<EntityName1>
<PropertyName1>“value of property1”</PropertyName1>
<PropertyName2>“value of property2”</PropertyName2>
<PropertyName3>“value of property1”</PropertyName3>
</EntityName1>
<EntityName2>
<PropertyName1>“value of property1”</PropertyName1>
<PropertyName2>“value of property2”</PropertyName2>
<PropertyName3>“value of property1”</PropertyName3>
</EntityName2>
</EntitySetName>
So, we have to re-structure the above REST-xml-structure, in order to get the structure that is expected by Integration Gateway.
In detail:
Rename the root node <results> to <StreetSet>
Rename the data nodes <result> to <Street>
Delete the info nodes <numFound> and <QTime>
Delete the attribute xmlns=”…” in the root node
Fortunately, the w3c.dom package provides support for modification of xml nodes.
My proposal for implementation:
def Document refactorDocument(Document document){
if(document == null){
log.logErrors(LogMessage.TechnicalError, "Could not load xml-document");
return;
}
//find nodes
Node resultsElement = document.getFirstChild();
NodeList childrenOfResults = resultsElement.getChildNodes();
// rename the root node: <results xmlns="http://gisgraphy.com">
document.renameNode(resultsElement, resultsElement.getNamespaceURI(), "StreetSet");
// remove xmlns-attribute from root node: example: <results xmlns="http://gisgraphy.com">
NamedNodeMap attributesMap = resultsElement.getAttributes();
if(attributesMap.getNamedItem("xmlns") != null){
try {
attributesMap.removeNamedItem("xmlns");
} catch (DOMException e) {
log.logErrors(LogMessage.TechnicalError, "Failed removing attribute.");
}
}
// nodes to delete:
Node numNode;
Node qtNode;
// rename all nodes of the REST service: <result>...
for(int i = 0; i < childrenOfResults.getLength(); i++){
Node childNode = childrenOfResults.item(i);
String nodeName = childNode.getNodeName();
if(nodeName.equals("numFound")){ // store this node for later deletion
numNode = childNode;
} else if(nodeName.equals("QTime")){ // store this node for later deletion
qtNode = childNode;
} else{ // rename this node
document.renameNode(childNode, childNode.getNamespaceURI(), "Street");
}
}
// delete 2 undesired nodes <numFound>50</numFound> and <QTime>..</QTime>
if(numNode != null){
resultsElement.removeChild(numNode);
}
if(qtNode != null){
resultsElement.removeChild(qtNode)
}
// done with refactoring
return document;
}
4. Step: Transform the Document to String
Now that we have manipulated the Document as desired, we’re ready to transform it back to String.
Here’s kind of default code, nothing to explain here:
def String transformToString(Document document, String encoding) {
// Explicitly check, otherwise method returns an XML Prolog
if (document == null) {
log.logErrors(LogMessage.TechnicalError, "Document is null.");
return null;
}
// create the transformer
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = null;
try {
transformer = transformerFactory.newTransformer();
} catch (TransformerConfigurationException e) {
log.logErrors(LogMessage.TechnicalError, "Failed creating Transformer.");
return null;
}
// configure the transformer
transformer.setOutputProperty(OutputKeys.METHOD, "xml");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
transformer.setOutputProperty(OutputKeys.ENCODING, encoding);
transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "4");
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
// prepare the output
OutputStream outputStream = new ByteArrayOutputStream();
OutputStreamWriter outputStreamWriter = null;
try {
outputStreamWriter = new OutputStreamWriter(outputStream, encoding);
} catch (UnsupportedEncodingException e) {
log.logErrors(LogMessage.TechnicalError, "Failed creating OutputStreamWriter");
return null;
}
// Finally do the transformation
BufferedWriter bufferedWriter = new BufferedWriter(outputStreamWriter);
try {
transformer.transform(new DOMSource(document), new StreamResult(bufferedWriter));
} catch (TransformerException e) {
log.logErrors(LogMessage.TechnicalError, "Transformation failed");
return null;
}
return outputStream.toString();
}
5. Step: Set the payload
The only thing that’s missing is to send the converted payload back to Integration Gateway.
This is done same way like in the previous tutorials, by setting the header.
Here’s now our convertPayload() method, that invokes all the other methods described above.
We invoke this method in the processResponseData() callback method.
def void convertPayload(Message message){
String payload = getResponsePayload(message);
//parse the payload and transform into Document
Document document = parsePayload(payload);
// now do the refactoring: throw away useless nodes from backend payload
document = refactorDocument(document);
String structuredXmlString = transformToString(document, "UTF-8");
if(structuredXmlString == null){
log.logErrors(LogMessage.TechnicalError, "Conversion failed");
//TODO proper error handling
return;
}
// finally
message.setBody(structuredXmlString);
message.setHeader("Content-Type", new String("xml"));
}
Final: Run
Note:
Our sample code is of course not enterprise-ready, but that’s normal for a tutorial.
More relevant is that our sample code doesn’t contain error handling, I mean, if the REST service doesn’t return the expected xml-payload, then we have to react appropriately and provide an empty feed, or a meaningful error message in the browser, we have to set the proper HTTP Status code, things like that.
I’m pasting the full script below, and I’m also attaching the relevant files to this blog post.
Note that the attached files are based on SP05. So if you use them for a different version, you have to adapt them, e.g. update the dependencies in the manifest file.
Now we can generate&deploy in Eclipse, and after configuring the destination, we can run and test the service.
Full source code
import java.nio.charset.StandardCharsets
import javax.xml.parsers.DocumentBuilder
import javax.xml.parsers.DocumentBuilderFactory
import javax.xml.parsers.ParserConfigurationException
import javax.xml.transform.OutputKeys
import javax.xml.transform.Transformer
import javax.xml.transform.TransformerConfigurationException
import javax.xml.transform.TransformerException
import javax.xml.transform.TransformerFactory
import javax.xml.transform.dom.DOMSource
import javax.xml.transform.stream.StreamResult
import org.w3c.dom.DOMException;
import org.w3c.dom.Document
import org.w3c.dom.NamedNodeMap;
import org.w3c.dom.Node
import org.w3c.dom.NodeList
import org.xml.sax.InputSource
import org.xml.sax.SAXException
import com.sap.gateway.ip.core.customdev.logging.LogMessage
import com.sap.gateway.ip.core.customdev.util.Message
/*
* ***************************
* FRAMEWORK CALLBACK METHODS
* ***************************
*/
/**
Function processRequestData will be called before the request data is
handed over to the REST service and can be used for providing
filter capabilities. User can manipulate the request data here.
*/
def Message processRequestData(message) {
return message;
}
/**
Function processResponseData will be called after the response data is received
from the REST service and is used to convert the REST response to OData
response using String APIs. User can manipulate the response data here.
*/
def Message processResponseData(message) {
convertPayload(message);
return message;
}
/*
* ***************************
* RESPONSE HANDLING
* ***************************
*/
def void convertPayload(Message message){
String payload = getResponsePayload(message);
//parse the payload and transform into Document
Document document = parsePayload(payload);
// now do the refactoring: throw away useless nodes from backend payload
document = refactorDocument(document);
String structuredXmlString = transformToString(document, "UTF-8");
if(structuredXmlString == null){
log.logErrors(LogMessage.TechnicalError, "Conversion failed");
//TODO proper error handling
return;
}
// finally
message.setBody(structuredXmlString);
message.setHeader("Content-Type", new String("xml"));
}
def String getResponsePayload(Message message){
String payload = message.getBody().toString();
//TODO do some checks on the payload here
return payload;
}
/**
* Parse the response body into a Document
* */
def Document parsePayload(String payload) {
if(payload == null || payload.length() < 1){
log.logErrors(LogMessage.TechnicalError, "Cannot parse empty payload.");
return null;
}
InputStream inputStream = new ByteArrayInputStream(payload.getBytes(StandardCharsets.UTF_8));
InputSource inputSource = new InputSource(inputStream);
DocumentBuilder parser;
try {
parser = DocumentBuilderFactory.newInstance().newDocumentBuilder();
return parser.parse(inputSource);
} catch (ParserConfigurationException e) {
log.logErrors(LogMessage.TechnicalError, "Parser creation failed");
return null;
} catch (SAXException e) {
log.logErrors(LogMessage.TechnicalError, "Response parsing failed");
return null;
}
}
def Document refactorDocument(Document document){
if(document == null){
log.logErrors(LogMessage.TechnicalError, "Failed to load document");
return;
}
//find nodes
Node resultsElement = document.getFirstChild();
NodeList childrenOfResults = resultsElement.getChildNodes();
// rename the root node: <results xmlns="http://gisgraphy.com">
document.renameNode(resultsElement, resultsElement.getNamespaceURI(), "StreetSet");
// remove xmlns-attribute from root node: example: <results xmlns="http://gisgraphy.com">
NamedNodeMap attributesMap = resultsElement.getAttributes();
if(attributesMap.getNamedItem("xmlns") != null){
try {
attributesMap.removeNamedItem("xmlns");
} catch (DOMException e) {
log.logErrors(LogMessage.TechnicalError, "Failed to remove attrib.");
}
}
// nodes to delete:
Node numNode;
Node qtNode;
// rename all nodes of the REST service: <result>...
for(int i = 0; i < childrenOfResults.getLength(); i++){
Node childNode = childrenOfResults.item(i);
String nodeName = childNode.getNodeName();
if(nodeName.equals("numFound")){ // store for later deletion
numNode = childNode;
} else if(nodeName.equals("QTime")){ // store for later deletion
qtNode = childNode;
} else{ // rename this node
document.renameNode(childNode, childNode.getNamespaceURI(), "Street");
}
}
// delete undesired nodes <numFound>50</numFound> and <QTime>..</QTime>
if(numNode != null){
resultsElement.removeChild(numNode);
}
if(qtNode != null){
resultsElement.removeChild(qtNode)
}
// done with refactoring
return document;
}
/**
* Transforms the specified document into a String representation.
* Removes the xml-declaration (<?xml version="1.0" ?>)
* @Param encoding should be UTF-8 in most cases
*/
def String transformToString(Document document, String encoding) {
if (document == null) {
log.logErrors(LogMessage.TechnicalError, "Document is null.");
return null;
}
// create the transformer
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = null;
try {
transformer = transformerFactory.newTransformer();
} catch (TransformerConfigurationException e) {
log.logErrors(LogMessage.TechnicalError, "Transformer creation failed.");
return null;
}
// configure the transformer
transformer.setOutputProperty(OutputKeys.METHOD, "xml");
transformer.setOutputProperty(OutputKeys.INDENT, "yes");
transformer.setOutputProperty(OutputKeys.ENCODING, encoding);
transformer.setOutputProperty("{http://xml.apache.org/xslt}indent-amount", "4");
transformer.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, "yes");
// prepare the output
OutputStream outputStream = new ByteArrayOutputStream();
OutputStreamWriter outputStreamWriter = null;
try {
outputStreamWriter = new OutputStreamWriter(outputStream, encoding);
} catch (UnsupportedEncodingException e) {
log.logErrors(LogMessage.TechnicalError, "OutputStreamWriter creation failed");
return null;
}
// Finally do the transformation
BufferedWriter bufferedWriter = new BufferedWriter(outputStreamWriter);
try {
transformer.transform(new DOMSource(document), new StreamResult(bufferedWriter));
} catch (TransformerException e) {
log.logErrors(LogMessage.TechnicalError, "Transformation failed.");
return null;
}
return outputStream.toString();
}
Further Reading
Preparing Eclipse for Groovy scripting:
http://scn.sap.com/docs/DOC-61719
Introduction in REST datasource part 1: Understanding the return structure in xml
Introduction in REST datasource part 2: Understanding the return structure in json
Introduction in REST datasource part 3: converting xml payload of a REST service for usage in Integration Gateway
Introduction in REST datasource part 4: implementing filter in custom script for Integration Gateway
Installing SMP toolkit:
Tutorial for Integration Gateway:
Tutorial about scripting in Integration Gateway:
- SAP Managed Tags:
