Skip to Content

This  blog is (even more) about Integration Gateway in SAP Mobile Platform 3.0 (SMP).

It is meant for those of you who have already created OData services based REST data source through Integration Gateway in SMP.

If you’ve implemented the script as described in my previous tutorial and if you’ve wondered if there isn’t a different way to convert the payload than just doing string operations… (only) then you should have a look at the following blog.

I’ve demonstrated it for XML – this blog will now do the same for JSON payload.

Note that this is not an official guide, just my personal findings and ways to do my stuff.

It might not meet all your requirements – so please do let me know your concerns or suggestions for improvements.

BTW, this tutorial is based on SMP SP05.

Prerequisites

  • My previous tutorial where we’ve created an OData service based on a REST data source
  • SAP Mobile Platform 3.0 SP 05
  • Eclipse Kepler with SAP Mobile Platform Tools installed

Preparation

SAP Mobile Platform

  • As described in one of my previous tutorials  you need to create a destination in your SMP server, that points to the host of the REST service that is used in this tutorial:
    http://sansad.co  with “no Authentication”
    Try the “Test Connection” button: the result should be green success popup.

    Note: you might need to enter proxy settings at https://localhost:8083/Admin/  -> Settings -> System

Eclipse

  • Create OData Implementation Project
  • Create OData model: We have to create an OData model that matches the REST service.

    Our model looks as follows:

    You can create the properties manually, or import the edmx file that is attached to this blog.
    Bind the QUERY operation to our example REST service: /api/legislators

  • Create Custom Code script for Groovy
    If desired, do the few preparation steps to enhance your development experience for Groovy, as explained here
  • Add dependency to gson in the MANIFEST.MF file

That’s it for the preparation.

Let’s start implementing the script.

Implementing the data structure conversion using JSON parser

Overview

We will be implementing only the method processResponseData()

The steps are:

  1. Get the payload string
  2. Parse the JSON payload
  3. Refactor
  4. Transform the JSON-object to String
  5. Set the converted payload string

1. Step: Get the payload

This is the same step a usual:

       String payload = message.getBody().toString();

2. Step: Parse the JSON payload

For parsing JSON, there are several libraries that could be used.

In this tutorial, we’re using com com.google.gson, the reason is that it is included in SMP, so we don’t have to install any own library/bundle to our SMP .

There are also several approaches for parsing JSON, one possibility is shown in the code below.

def JsonElement parsePayload(String payload){

     JsonParser parser = new JsonParser();

 

     return parser.parse(payload);

}

3. Step: Refactor the JSON Elements

Quick recap: what do we have to do?

This is the response body that we get from the REST service (http://sansad.co/api/legislators):

And this is the structure that is expected/required by Integration Gateway:

{

“results”:

     [

          {

               “PropertyName1”:“PropertyValue1”,

               “PropertyName2”:“PropertyValue2”

          }

     ]

}

So, we have to re-structure the above REST-json-structure, in order to get the structure that is expected by Integration Gateway.

In detail:

  1. Delete the undesired (informative) nodes count and page
  2. Remove all the properties that we don’t have in our OData model

We use API that is provided by com.google.gson:

def JsonObject refactorJsonElement(JsonElement jsonElement){

 

     JsonObject feedJsonObject = jsonElement.getAsJsonObject(); // e.g. {“results”:[{“age”:60,”…

 

     //first refactoring

     doRefactoringForFeed(feedJsonObject);

 

     // second refactoring: need to remove some properties, because the aren’t defined in our OData model

     JsonArray resultsArray = feedJsonObject.get(“results”).getAsJsonArray(); // it is a feed (array)

 

     // we’re performing a QUERY operation, so loop over all entries and refactor each

     for(int i = 0; i < resultsArray.size(); i++){

          JsonElement entryElement = resultsArray.get(i);

          if(entryElement instanceof JsonObject){

               //remove undesired properties, corresponding to edmx

               doRefactoringForEntry((JsonObject)entryElement);

          }

     }

     return feedJsonObject;

}

  1. In a first step, we get the top-level node, that is the feed, and remove the 2 undesired child nodes.
    These 2 nodes are for information purpose and don’t match to data structure

    And this is how it is removed:

    def JsonObject doRefactoringForFeed(JsonObject feedJsonObject){
           feedJsonObject.remove(“count”);
           feedJsonObject.remove(“page”);

                return feedJsonObject;
    }

  2. In a second step, we ask the feed node for the list of entries, loop over them and for each entry, we remove all the undesired properties.
    The “undesired properties” are those that are in the REST-service, but not in our OData model.
    In the following screenshot, only the desired properties are marked red:

    To make the sample code more comprehensive, I’ve hard-coded the  names of the undesired properties.
    (There’s also a way to do it generically, I’ll publish it in a separate blog.)

    def JsonObject doRefactoringForEntry(JsonObject entryObject){
        //remove undesired properties, corresponding to edmx
        entryObject.remove(“age”);
        entryObject.remove(“attendance_percentage”);
        entryObject.remove(“constituency”);
        entryObject.remove(“debates”);
        entryObject.remove(“education_details”);
        entryObject.remove(“education_qualification”);
        entryObject.remove(“elected”);
        entryObject.remove(“gender”);
        entryObject.remove(“house”);
        entryObject.remove(“in_office”);
        entryObject.remove(“last_name”);
        entryObject.remove(“notes”);
        entryObject.remove(“private_bills”);
        entryObject.remove(“questions”);
        entryObject.remove(“term_end”);
        entryObject.remove(“term_start”);

        return entryObject;
    }

4. Step: Transform the JSON object to String

Now that we have manipulated the JSON nodes as desired, we’re ready to transform it back to string.

def String transformToString(JsonObject jsonObject){

   

       return jsonObject.toString();

}

5. Step: Set the payload

The only thing that’s missing is to send the converted payload back to the Integration Gateway.

This is done same way like in the previous tutorials, by setting the header.

Here’s now our convertPayload() method, that invokes all the other methods described above.

We invoke this method in the processResponseData() callback method.

def void convertPayload(Message message){

   

     String payload = getResponsePayload(message);

     //parse the payload and transform into JsonObject

     JsonElement jsonElement = parsePayload(payload);

     // now do the refactoring: throw away useless nodes from backend payload

     JsonObject jsonObject = refactorJsonElement(jsonElement);

     // transform back to string

     String structuredJsonString = transformToString(jsonObject);

   

     if(structuredJsonString == null){

          log.logErrors(LogMessage.TechnicalError, “Error: failed to convert the backend payload to structured XML”);

          //TODO proper error handling

          return;

     }

   

     // finally

     message.setBody(structuredJsonString);

     message.setHeader(“Content-Type”, new String(“json”));

}

Final: Run

Now we can generate&deploy in Eclipse, and after configuring the destination, we can run and test the service.

Note:

Our sample code is of course not enterprise-ready, but that’s normal of course.

More relevant is that our sample code doesn’t contain error handling, I mean, if the REST service doesn’t return the expected xml-payload, then we have to react appropriately and provide an empty feed, or a meaningful error message in the browser, set the proper HTTP Status code, things like that.

I’m pasting the full script below, and I’m also attaching the relevant files to this blog post.

Full source code

import com.google.gson.JsonArray

import com.google.gson.JsonElement

import com.google.gson.JsonObject

import com.google.gson.JsonParser

import com.sap.gateway.ip.core.customdev.logging.LogMessage

import com.sap.gateway.ip.core.customdev.util.Message

/*

* ***************************

*  FRAMEWORK CALLBACK METHODS

* ***************************

*/

/**

  Function processRequestData will be called before the request data is

  handed over to the REST service and can be used for providing

  filter capabilities. User can manipulate the request data here.

*/

def Message processRequestData(message) {

     return message;

}

/**

  Function processResponseData will be called after the response data is received

  from the REST service and is used to convert the REST response to OData

  response using String APIs. User can manipulate the response data here.

*/

def Message processResponseData(message) {

     convertPayload(message);

     return message;

}

/*

* ***************************

*  RESPONSE HANDLING

* ***************************

*/

def void convertPayload(Message message){

   

     String payload = getResponsePayload(message);

     //parse the payload and transform into JsonObject

     JsonElement jsonElement = parsePayload(payload);

     // now do the refactoring: throw away useless nodes from backend payload

     JsonObject jsonObject = refactorJsonElement(jsonElement);

     // transform back to string

     String structuredJsonString = transformToString(jsonObject);

   

     if(structuredJsonString == null){

          log.logErrors(LogMessage.TechnicalError, “Error: failed to convert the backend payload to structured XML”);

          //TODO proper error handling

          return;

     }

   

     // finally

     message.setBody(structuredJsonString);

     message.setHeader(“Content-Type”, new String(“json”));

}

def String getResponsePayload(Message message){

     String payload = message.getBody().toString();

   

     //TODO do some checks on the payload here

   

     return payload;

}

def JsonElement parsePayload(String payload){

     JsonParser parser = new JsonParser();

   

     return parser.parse(payload);

}

def JsonObject refactorJsonElement(JsonElement jsonElement){

   

     JsonObject feedJsonObject = jsonElement.getAsJsonObject(); // {“results”:[{“age”:60,”…

   

     //first refactoring

     doRefactoringForFeed(feedJsonObject);

   

     // second refactoring: need to remove some properies, because the aren’t defined in our odata model

     JsonArray resultsArray = feedJsonObject.get(“results”).getAsJsonArray(); // it is a feed / array

   

     // we’re performing a QUERY operation, so loop over all entries and refactor each

     for(int i = 0; i < resultsArray.size(); i++){

          JsonElement entryElement = resultsArray.get(i);

          if(entryElement instanceof JsonObject){

               //remove undesired properties, corresponding to edmx

               doRefactoringForEntry((JsonObject)entryElement);

          }

     }

     return feedJsonObject;

}

def JsonObject doRefactoringForFeed(JsonObject feedJsonObject){

     feedJsonObject.remove(“count”);

     feedJsonObject.remove(“page”);

   

     return feedJsonObject;

}

   

   

def JsonObject doRefactoringForEntry(JsonObject entryObject){

     //remove undesired properties, corresponding to edmx

     entryObject.remove(“age”);

     entryObject.remove(“attendance_percentage”);

     entryObject.remove(“constituency”);

     entryObject.remove(“debates”);

     entryObject.remove(“education_details”);

     entryObject.remove(“education_qualification”);

     entryObject.remove(“elected”);

     entryObject.remove(“gender”);

     entryObject.remove(“house”);

     entryObject.remove(“in_office”);

     entryObject.remove(“last_name”);

     entryObject.remove(“notes”);

     entryObject.remove(“private_bills”);

     entryObject.remove(“questions”);

     entryObject.remove(“term_end”);

     entryObject.remove(“term_start”);

   

     return entryObject;

}

def String transformToString(JsonObject jsonObject){

   

     return jsonObject.toString();

}

Further Reading

Preparing Eclipse for Groovy scripting:

http://scn.sap.com/docs/DOC-61719

Introduction in REST datasource part 1: Understanding the return structure in xml

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/10/understanding-rest-data-source-in-integration-gateway-1-query-very-simplified

Introduction in REST datasource part 2: Understanding the return structure in json

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/11/integration-gateway-understanding-rest-data-source-2-query–json-very-simplified

Introduction in REST datasource part 3: converting xml payload of a REST service for usage in Integration Gateway

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/12/integration-gateway-understanding-rest-data-source-3-query–xml-standard

Introduction in REST datasource part 4: implementing filter in custom script for Integration Gateway

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/17/integration-gateway-understanding-rest-data-source-4-filtering

Introduction in REST datasource part 5: converting xml payload using xml parser instead of string operations

http://scn.sap.com/community/developer-center/mobility-platform/blog/2015/02/18/integration-gateway-understanding-rest-data-source-5-using-xml-parser

Installing SMP toolkit:

http://scn.sap.com/community/developer-center/mobility-platform/blog/2014/08/22/how-to-install-sap-mobile-platform-tools-for-integration-gateway-in-eclipse-kepler

Tutorial for Integration Gateway:

http://scn.sap.com/community/developer-center/mobility-platform/blog/2014/06/10/creating-an-odata-service-based-on-sap-gateway-soap-jdbc-and-jpa-data-sources-ba

Tutorial about scripting in Integration Gateway:

http://scn.sap.com/community/developer-center/mobility-platform/blog/2014/10/17/integration-gateway-using-custom-scripting-to-map-entity-sets-to-data-sources

To report this post you need to login first.

2 Comments

You must be Logged on to comment or reply to a post.

  1. Vishnu Pankajakshan Panicker

    Hi Carlos,

    The tutorial is awesome 🙂   bookmarked..!!!!!

    Sorry that i ask u this question… just curious 🙂

    Is the intention was to covert the Response into JSON…??? Or its an another method to manipulate the Payload????

    Regards,

    Vishnu

    (0) 
    1. Carlos Roggan Post author

      Hi Vishnu,

      thanks so much! I’m so glad you liked it !!!

      Please never hesitate to ask questions… that’s why we have this forum 😉

      Maybe in this tutorial I should say it more clearly instead of pointing to previous tutorials.

      The point is:

      You’re fetching your desired data from a data source that is  a REST service.

      This REST service returns the data in json format

      The above example is http://sansad.co/api/legislators

      (Depending on the REST service that you use, it might be possible to define your desired format in a url parameter, but sometimes you have to live with the format that the REST service delivers)

      Now in your script you have to manipulate the returned payload such that is gets a structure that can be understood by the Integration Gateway Framework.

      The Gateway Framework can understand both xml and json.

      Now, in order to manipulate the json payload, typically string operations are used (in demos and docus), as it is shorter.

      In this blog I wanted to show an alternative way, using a parser.

      Regards,

      Carlos

      (0) 

Leave a Reply