Skip to Content

This  blog is about Integration Gateway in SAP Mobile Platform 3.0 (SMP).

When you create an OData service based on Integration Gateway (IGW), connecting to REST data source, and you deploy it to SMP, you can invoke a collection and you can apply a system query option like $select, it will work out of the box without any further scripting in your custom code.

But there’s no support for $top out of the box.

However, it can be achieved via manual coding in your script.

In one of my previous tutorials, I’ve explained how you can add $filter functionality to your OData service by adding the required implementation to your custom code.

In that tutorial, the procedure was to modify the URI of the REST service, before it gets called.

Thus, we added the code in the processRequestData() method.

The URI was modified in order to call the backend-REST-service with the (adapted) URI that includes the filter in the syntax that the REST-service supports.

So this procedure makes only sense if the backend REST service does support filtering.

In the present tutorial, we cannot apply the same procedure for the $top, because our backend-REST service doesn’t support a similar functionality.

Note:

if you’re using a backend-REST-service that does support a kind of $top functionality, then you would proceed similar like described in the filter-tutorial:
modify the URI in the processRequestData() method such that it complies to the requirements of the backend-REST-service.

In the present tutorial, we proceed differently:

we implement the $top manually in our script, since we cannot delegate to the backend-REST-service.

This tutorial is based on SMP SP08, but it should be working fine with other versions of SMP

Note: the code can be found attached to this blog.

Table of contents

1. Prerequisites

2. Preparation

3. Create a Service based on REST data source

   3.1. OData model

   3.2. Bind data source for query operation

   3.3. Custom code

   3.4. Test the query operation

4. Implement $top

   4.1. Overview

   4.2. Implement the $top

   4.3. Test the $top

5. Summary

6. Links

7. Appendix

1. Prerequisites

If you’re new to IGW and REST data source, I’d recommend that you go through some of my previous tutorials , explaining REST data source, QUERY  and READ  operation.

In any case, you should have checked the tutorial that uses an xml parser for manipulating the response payload:

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/18/integration-gateway-understanding-rest-data-source-5-using-xml-parser

Please check the Links section at the bottom for more info.

Furthermore, you need:

  • Eclipse with SAP Mobile Platform Tools installed
  • SAP Mobile Platform
  • Basic knowledge about OData provisioning using the Integration Gateway component of SMP

2. Preparation

REST Service

For this tutorial, we’re using a REST service that is public, you only need to sign up, afterwards you can access it with your SCN user and password.

Please see the following document for details:

Sign up and get started with the SAP Gateway – Demo Consumption System

Finally, you should be able to access it via the following URL:

https://sapes1.sapdevcenter.com/sap/opu/rest/address/companies

Destination

In your SMP, you need to create an HTTP destination to the following URL:

https://sapes1.sapdevcenter.com/



/wp-content/uploads/2015/08/destination_774613.jpg

Please refer to the Appendix section for some hints about configuring the destination

3. Create the Service based on REST data source

3.1. OData Model


Define OData model

The backend REST service that we’re using provides the following sample data:

So for our OData model, we create the following Entity Type:

3.2. Bind data source for query operation

We bind the entityset to REST data source for the QUERY operation and provide the following relative URI:


/sap/opu/rest/address/companies

3.3. Custom Code

We create the custom code for the QUERY operation, choosing Groovy as language.

The generated file content has to be replaced with the following (note that this code works for SMP SP08)

import java.nio.charset.StandardCharsets

import javax.xml.parsers.DocumentBuilder

import javax.xml.parsers.DocumentBuilderFactory

import javax.xml.parsers.ParserConfigurationException

import javax.xml.transform.OutputKeys

import javax.xml.transform.Transformer

import javax.xml.transform.TransformerConfigurationException

import javax.xml.transform.TransformerException

import javax.xml.transform.TransformerFactory

import javax.xml.transform.dom.DOMSource

import javax.xml.transform.stream.StreamResult

import org.apache.olingo.odata2.api.uri.KeyPredicate

import org.apache.olingo.odata2.api.uri.NavigationSegment

import org.apache.olingo.odata2.api.uri.UriInfo

import org.w3c.dom.Document

import org.w3c.dom.Node

import org.w3c.dom.NodeList

import org.xml.sax.InputSource

import org.xml.sax.SAXException

import com.sap.gateway.ip.core.customdev.logging.ILogger

import com.sap.gateway.ip.core.customdev.logging.LogMessage

import com.sap.gateway.ip.core.customdev.util.Message

def Message processRequestData(message) {

       return message;

}

def Message processResponseData(message) {

    message = (Message)message;

    String bodyString = (String) message.getBody();

       /* CONVERT PAYLOAD */

    InputSource inputSource = new InputSource(new ByteArrayInputStream(

                                        bodyString.getBytes(StandardCharsets.UTF_8)));

    inputSource.setEncoding(“UTF-8”); // REQUIRED, input has a BOM, can’t be parsed

    Document document = loadXMLDoc(inputSource);

       // now do the refactoring: throw away useless nodes from backend payload

    document = refactorDocument(document);

       // convert the modified DOM back to string

    String structuredXmlString = toStringWithoutProlog(document, “UTF-8”);

       /* FINALLY */

    message.setBody(structuredXmlString);

       return message;

}

def Document loadXMLDoc(InputSource source) {

    DocumentBuilder parser;

       try {

       parser = DocumentBuilderFactory.newInstance().newDocumentBuilder();

    } catch (ParserConfigurationException e) {

       ((ILogger)log).logErrors(LogMessage.TechnicalError, “Error while creating parser”);

       return null;

    }

       // now parse

       try {

             return parser.parse(source);

    } catch (SAXException e) {

       ((ILogger)log).logErrors(LogMessage.TechnicalError, “Error parsing source”);

            return null;

    }

}

def Document refactorDocument(Document document){

       // find nodes

    Node rootElement = document.getFirstChild();

    Node asxValuesNode = rootElement.getFirstChild();

    Node proddataNode = asxValuesNode.getFirstChild();

    NodeList snwdNodeList = proddataNode.getChildNodes();

       //rename all nodes of the feed

    document.renameNode(proddataNode, proddataNode.getNamespaceURI(), “Companies”);

       for(int i = 0; i < snwdNodeList.getLength(); i++){

       Node snwdNode = snwdNodeList.item(i);

       document.renameNode(snwdNode, snwdNode.getNamespaceURI(), “Company”);

    }

       //replace node

    Node cloneNode = proddataNode.cloneNode(true);

    document.replaceChild(cloneNode, rootElement);

       return document;

}

/**

* Transforms the specified document into a String representation.

* Removes the xmldeclaration (<?xml version=”1.0″ ?>)

* @param encoding should be UTF8 in most cases

*/

def String toStringWithoutProlog(Document document, String encoding) {

       // Explicitly check this; otherwise this method returns just an XML Prolog

       if (document == null) {

       ((ILogger)log).logErrors(LogMessage.TechnicalError, “Error: document is null.”);

            return null;

    }

       TransformerFactory transformerFactory = TransformerFactory.newInstance();

       Transformer t = null;

       try {

       t = transformerFactory.newTransformer();

    } catch (TransformerConfigurationException e) {

       ((ILogger)log).logErrors(LogMessage.TechnicalError, “Error creating Transformer”);

             return null;

    }

    t.setOutputProperty(OutputKeys.METHOD, “xml”);

    t.setOutputProperty(OutputKeys.INDENT, “yes”);

    t.setOutputProperty(OutputKeys.ENCODING, encoding);

    t.setOutputProperty(“{http://xml.apache.org/xslt}indent-amount”, “4”);

    t.setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, “yes”);

    OutputStream os = new ByteArrayOutputStream();

    OutputStreamWriter osw = null;

       try {

       osw = new OutputStreamWriter(os, encoding);

    } catch (UnsupportedEncodingException e) {

       ((ILogger)log).logErrors(LogMessage.TechnicalError, “Error creating Writer”);

             return null;

    }

    BufferedWriter bw = new BufferedWriter(osw);

       try {

       t.transform(new DOMSource(document), new StreamResult(bw));

    } catch (TransformerException e) {

       ((ILogger)log).logErrors(LogMessage.TechnicalError, “Error during transformation”);

            return null;

    }

       return os.toString();

}

3.4. Test the QUERY operation

Deploy the project and try it. The following request should give a valid response:


https://localhost:8083/gateway/odata/SAP/REST_DEMO_TEST;v=1/Companies

4. Implement $top

Until now, everything was preparation only.

Now that we have a working service, let’s have a closer look into the actual topic of this Blog:

Implementing the $top

4.1. Overview

The code the we’ve created up to here has already been explained in detail in the respective tutorial

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/18/integration-gateway-understanding-rest-data-source-5-using-xml-parser

What it does is:

  1. Get the payload string
  2. Parse the xml / convert to DOM
  3. Refactor the DOM
  4. Transform the DOM to String
  5. Set the converted payload string to the message object

This gives us a chance to hook in.

What do we want to do?

The “normal” QUERY operation provides a number of entries.

When adding the $top to the URL, we want to see only a reduced number of entries.

Example:

The result of the following URL is a list with 2 entries:

https://localhost:8083/gateway/odata/SAP/REST_DEMO_TEST;v=1/Companies?$top=2

These 2 entries are the first 2 entries of the “normal” QUERY.

How to implement it?

After step “3. Refactor the DOM”, we have a well-structured list with all entries.

From this list we have to take the first 2 and remove all the rest.

The remaining steps 4 and 5 can remain untouched.

So our new script that supports $top will have the following sequence:

  1. Get the payload string
  2. Parse the xml / convert to DOM
  3. Refactor the DOM
  4. Apply $top
  5. Transform the DOM to String
  6. Set the converted payload string to the message object

4.2. Implement the $top

We create an additional method which will contain the implementation for handling the $top

First we have to retrieve the value of the $top expression that has been provided by the user.

If the user has invoked the following URL

https://localhost:8083/gateway/odata/SAP/REST_DEMO_TEST;v=1/Companies?$top=2

Then we need the value ‘2’

This is retrieved from the UriInfo object.

The UriInfo object can be accessed from the headers

The UriInfo instance can be asked for the $top number

It returns null if the user doesn’t specify the $top option.

This has to be considered in our code


UriInfo uriInfo = (UriInfo) message.getHeaders().get(“UriInfo”);

Integer topOption = uriInfo.getTop();

if(topOption == null){

       return;

}

int topNumber = topOption.intValue();

Now that we know how many entries (starting from the beginning) have to be displayed, we need to adapt the payload to it.

Our method receives the Document instance which is the DOM representation of the response of the backend-REST-service.

Since it is already refactored, it contains the full payload in the following structure:

<Companies>

    <Company>

          <ID>“1”</ID>

          <NAME>“SAP”</NAME>

          <STREET>“Dietmar-Hopp-Allee”</STREET>

           …

    </Company>

    <Company>

          <ID>“2”</ID>

          <NAME>“<companyName>”</NAME>

          <STREET>“<companyStreet>”</STREET>

           …

    </Company>

    …

</Companies>



The first node is the parent node for a list of (Company-) nodes that has to be modified according to the specified $top value.

So what we have to do is to delete all child nodes, but leave the first ones.

One possible solution is:

Check if the number of existing entries is higher the the $top

Delete the last node

Check again

Delete the last node

And so on

while(entitySetNode.getChildNodes().getLength() > topNumber){

       Node lastCompany = entitySetNode.getLastChild();

       entitySetNode.removeChild(lastCompany);

}



Furthermore, some details to consider:

$top=-1

What if the value that the user has given in the $top expression is invalid?

This is handled by the Olingo library, we don’t need to take care.

$top=0

This is a valid number, we return an empty list

Instead of deleting each child node one by one, we replace the current root node with an empty one

$top=9999

If the user specifies a number that is too high, then we can just return the full list.

It is up to the service implementor to decide if he wants to throw an exception for such a case.

And this is how our helper method looks like:

def applyTop(Document document, Message message){

    UriInfo uriInfo = (UriInfo) message.getHeaders().get(“UriInfo”);

    Integer topOption = uriInfo.getTop();

       if(topOption == null){

             return;

    }

       int topNumber = topOption.intValue();

    Node entitySetNode = document.getFirstChild();

    NodeList entities = entitySetNode.getChildNodes();

       if(topNumber > entities.getLength()){

             return;

    }else if (topNumber == 0){

       document.replaceChild(document.createElement(“Companies”), entitySetNode);

    }else{

            while(entitySetNode.getChildNodes().getLength() > topNumber){

            Node lastCompany = entitySetNode.getLastChild();

            entitySetNode.removeChild(lastCompany);

       }

    }

}




At the end, we have to call our new applyTop()-method.

We invoke it in the processRequestData(), after we’ve refactored the payload:


def Message processResponseData(message) {

    message = (Message)message;

    String bodyString = (String) message.getBody();

       /* CONVERT PAYLOAD */

    InputSource inputSource = new InputSource(new ByteArrayInputStream(

                                        bodyString.getBytes(StandardCharsets.UTF_8)));

    inputSource.setEncoding(“UTF-8”); // REQUIRED, input has BOM, can’t be parsed

    Document document = loadXMLDoc(inputSource);

       // now do the refactoring: throw away useless nodes from backend payload

    document = refactorDocument(document);

       // handle system query options

    applyTop(document, message);

       // convert the modified DOM back to string

    String structuredXmlString = toStringWithoutProlog(document, “UTF-8”);

       /* FINALLY */

    message.setBody(structuredXmlString);

       return message;

}



4.3. Test the $top

Deploy the project and try the following URLS:



5. Summary

In this tutorial we’ve learned how to implement the system query option $top.

Since the backend REST service doesn’t support a kind of $top functionality, we had to go for the approach to fetch the full data from the backend and then remove the unneeded entries in our script.

From performance perspective, this is not ideal, but if the backend REST service doesn’t support it, it is the only way to achieve it.

6. Links

Installing SMP Toolkit:

http://scn.sap.com/community/developer-center/mobility-platform/blog/2014/08/22/how-to-install-sap-mobile-platform-tools-for-integration-gateway-in-eclipse-kepler

Tutorial for OData provisioning in SMP:

http://scn.sap.com/community/developer-center/mobility-platform/blog/2014/06/10/creating-an-odata-service-based-on-sap-gateway-soap-jdbc-and-jpa-data-sources-ba

Preparing Eclipse for Groovy scripting: http://scn.sap.com/docs/DOC-61719

Introduction in REST datasource part 1: Understanding the return structure in xml

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/10/understanding-rest-data-source-in-integration-gateway-1-query-very-simplified

Introduction in REST data source part 3: Implementing the QUERY operation

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/12/integration-gateway-understanding-rest-data-source-3-query–xml-standard

How to use XML parser to convert the payload of the REST service

Integration Gateway: Understanding REST data source [5]: using XML parser

Implement $filter in your OData service, based on REST data source

Integration Gateway: Understanding REST data source [4]: Filtering

Overview of all REST blogs

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/04/08/integration-gateway-rest-data-source-overview-of-blogs

The official documentation: http://help.sap.com/mobile-platform/




7. Appendix

Configuring the destination

The URL for the Destination:

https://sapes1.sapdevcenter.com

Since the URL is HTTPS, you need to download the certificate and import it into your SMP keyStore.

Test Connection:

For this Destination, it isn’t possible to do a “Test Connection”, as the server doesn’t send a valid response for the configured URL

As a workaround, you can proceed as follows:

Create a second destination, which is only used to test if the target host can be reached.

This second destination points to a URL that actually can send a valid response.

For example, enter the following URL as destination URL:

https://sapes1.sapdevcenter.com/sap/opu/rest/address/companies

As such, you use this second destination to do the “Test Connection”.

If this succeeds, then the first destination should be fine as well.

This first destination will be used to configure our OData service, that we create in this tutorial.

Proxy:

If you get an error message on connection test, you might consider the following:

You might need to enter proxy settings in your SMP:

https://localhost:8083/Admin/ -> Settings-> System

Note that you might need to restart the SMP server after changing the proxy settings.




To report this post you need to login first.

2 Comments

You must be Logged on to comment or reply to a post.

  1. Bjoern Woppmann

    Thanks for your continuous efforts on showcasing the capabilities of custom scripting with the REST data source in the SAP Mobile Platform. It really helps to see the examples you provide.

    (0) 

Leave a Reply