Implementing the data structure conversion using JSON parser

CarlosRoggan · ‎02-19-2015

This blog is (even more) about Integration Gateway in SAP Mobile Platform 3.0 (SMP).

It is meant for those of you who have already created OData services based REST data source through Integration Gateway in SMP.

If you’ve implemented the script as described in my previous tutorial and if you’ve wondered if there isn’t a different way to convert the payload than just doing string operations… (only) then you should have a look at the following blog.

I’ve demonstrated it for XML – this blog will now do the same for JSON payload.

Note that this is not an official guide, just my personal findings and ways to do my stuff.

It might not meet all your requirements – so please do let me know your concerns or suggestions for improvements.

BTW, this tutorial is based on SMP SP05.

Prerequisites

My previous tutorial where we’ve created an OData service based on a REST data source
SAP Mobile Platform 3.0 SP 05
Eclipse Kepler with SAP Mobile Platform Tools installed

Preparation

SAP Mobile Platform

As described in one of my previous tutorials you need to create a destination in your SMP server, that points to the host of the REST service that is used in this tutorial:
http://sansad.co with "no Authentication"
Try the “Test Connection” button: the result should be green success popup.

Note: you might need to enter proxy settings at https://localhost:8083/Admin/ -> Settings -> System

Eclipse

Create OData Implementation Project
Create OData model: We have to create an OData model that matches the REST service.

Our model looks as follows:

You can create the properties manually, or import the edmx file that is attached to this blog.
Bind the QUERY operation to our example REST service: /api/legislators
Create Custom Code script for Groovy
If desired, do the few preparation steps to enhance your development experience for Groovy, as explained here
Add dependency to gson in the MANIFEST.MF file

That’s it for the preparation.

Let’s start implementing the script.

Implementing the data structure conversion using JSON parser

Overview

We will be implementing only the method processResponseData()

The steps are:

Get the payload string
Parse the JSON payload
Refactor
Transform the JSON-object to String
Set the converted payload string

1. Step: Get the payload

This is the same step a usual:

String payload = message.getBody().toString();

2. Step: Parse the JSON payload

For parsing JSON, there are several libraries that could be used.

In this tutorial, we’re using com com.google.gson, the reason is that it is included in SMP, so we don’t have to install any own library/bundle to our SMP .

There are also several approaches for parsing JSON, one possibility is shown in the code below.

def JsonElement parsePayload(String payload){

JsonParser parser = new JsonParser();

return parser.parse(payload);

}

3. Step: Refactor the JSON Elements

Quick recap: what do we have to do?

This is the response body that we get from the REST service (http://sansad.co/api/legislators😞

And this is the structure that is expected/required by Integration Gateway:

{
"results":
     [
          {
               "PropertyName1":"PropertyValue1",
               "PropertyName2":"PropertyValue2"
          }
     ]
}

So, we have to re-structure the above REST-json-structure, in order to get the structure that is expected by Integration Gateway.

In detail:

Delete the undesired (informative) nodes count and page
Remove all the properties that we don’t have in our OData model

We use API that is provided by com.google.gson:

def JsonObject refactorJsonElement(JsonElement jsonElement){

JsonObject feedJsonObject = jsonElement.getAsJsonObject(); // e.g. {"results":[{"age":60,"...

//first refactoring

doRefactoringForFeed(feedJsonObject);

// second refactoring: need to remove some properties, because the aren't defined in our OData model

JsonArray resultsArray = feedJsonObject.get("results").getAsJsonArray(); // it is a feed (array)

// we're performing a QUERY operation, so loop over all entries and refactor each

for(int i = 0; i < resultsArray.size(); i++){

JsonElement entryElement = resultsArray.get(i);

if(entryElement instanceof JsonObject){

//remove undesired properties, corresponding to edmx

doRefactoringForEntry((JsonObject)entryElement);

}

return feedJsonObject;

}

In a first step, we get the top-level node, that is the feed, and remove the 2 undesired child nodes.
These 2 nodes are for information purpose and don't match to data structure

And this is how it is removed:

def JsonObject doRefactoringForFeed(JsonObject feedJsonObject){
       feedJsonObject.remove("count");
       feedJsonObject.remove("page");

            return feedJsonObject;
}
In a second step, we ask the feed node for the list of entries, loop over them and for each entry, we remove all the undesired properties.
The “undesired properties” are those that are in the REST-service, but not in our OData model.
In the following screenshot, only the desired properties are marked red:

To make the sample code more comprehensive, I’ve hard-coded the names of the undesired properties.
(There’s also a way to do it generically, I’ll publish it in a separate blog.)

def JsonObject doRefactoringForEntry(JsonObject entryObject){
    //remove undesired properties, corresponding to edmx
    entryObject.remove("age");
    entryObject.remove("attendance_percentage");
    entryObject.remove("constituency");
    entryObject.remove("debates");
    entryObject.remove("education_details");
    entryObject.remove("education_qualification");
    entryObject.remove("elected");
    entryObject.remove("gender");
    entryObject.remove("house");
    entryObject.remove("in_office");
    entryObject.remove("last_name");
    entryObject.remove("notes");
    entryObject.remove("private_bills");
    entryObject.remove("questions");
    entryObject.remove("term_end");
    entryObject.remove("term_start");

    return entryObject;
}

4. Step: Transform the JSON object to String

Now that we have manipulated the JSON nodes as desired, we’re ready to transform it back to string.

def String transformToString(JsonObject jsonObject){

return jsonObject.toString();

}

5. Step: Set the payload

The only thing that’s missing is to send the converted payload back to the Integration Gateway.

This is done same way like in the previous tutorials, by setting the header.

Here’s now our convertPayload() method, that invokes all the other methods described above.

We invoke this method in the processResponseData() callback method.

def void convertPayload(Message message){

String payload = getResponsePayload(message);

//parse the payload and transform into JsonObject

JsonElement jsonElement = parsePayload(payload);

// now do the refactoring: throw away useless nodes from backend payload

JsonObject jsonObject = refactorJsonElement(jsonElement);

// transform back to string

String structuredJsonString = transformToString(jsonObject);

if(structuredJsonString == null){

log.logErrors(LogMessage.TechnicalError, "Error: failed to convert the backend payload to structured XML");

//TODO proper error handling

return;

}

// finally

message.setBody(structuredJsonString);

message.setHeader("Content-Type", new String("json"));

}

Final: Run

Now we can generate&deploy in Eclipse, and after configuring the destination, we can run and test the service.

Note:

Our sample code is of course not enterprise-ready, but that’s normal of course.

More relevant is that our sample code doesn’t contain error handling, I mean, if the REST service doesn’t return the expected xml-payload, then we have to react appropriately and provide an empty feed, or a meaningful error message in the browser, set the proper HTTP Status code, things like that.

I’m pasting the full script below, and I’m also attaching the relevant files to this blog post.

Full source code

import com.google.gson.JsonArray

import com.google.gson.JsonElement

import com.google.gson.JsonObject

import com.google.gson.JsonParser

import com.sap.gateway.ip.core.customdev.logging.LogMessage

import com.sap.gateway.ip.core.customdev.util.Message

/*

* ***************************

* FRAMEWORK CALLBACK METHODS

* ***************************

*/

/**

Function processRequestData will be called before the request data is

handed over to the REST service and can be used for providing

filter capabilities. User can manipulate the request data here.

*/

def Message processRequestData(message) {

return message;

}

/**

Function processResponseData will be called after the response data is received

from the REST service and is used to convert the REST response to OData

response using String APIs. User can manipulate the response data here.

*/

def Message processResponseData(message) {

convertPayload(message);

return message;

}

/*

* ***************************

* RESPONSE HANDLING

* ***************************

*/

def void convertPayload(Message message){

String payload = getResponsePayload(message);

//parse the payload and transform into JsonObject

JsonElement jsonElement = parsePayload(payload);

// now do the refactoring: throw away useless nodes from backend payload

JsonObject jsonObject = refactorJsonElement(jsonElement);

// transform back to string

String structuredJsonString = transformToString(jsonObject);

if(structuredJsonString == null){

log.logErrors(LogMessage.TechnicalError, "Error: failed to convert the backend payload to structured XML");

//TODO proper error handling

return;

}

// finally

message.setBody(structuredJsonString);

message.setHeader("Content-Type", new String("json"));

}

def String getResponsePayload(Message message){

String payload = message.getBody().toString();

//TODO do some checks on the payload here

return payload;

}

def JsonElement parsePayload(String payload){

JsonParser parser = new JsonParser();

return parser.parse(payload);

}

def JsonObject refactorJsonElement(JsonElement jsonElement){

JsonObject feedJsonObject = jsonElement.getAsJsonObject(); // {"results":[{"age":60,"...

//first refactoring

doRefactoringForFeed(feedJsonObject);

// second refactoring: need to remove some properies, because the aren't defined in our odata model

JsonArray resultsArray = feedJsonObject.get("results").getAsJsonArray(); // it is a feed / array

// we're performing a QUERY operation, so loop over all entries and refactor each

for(int i = 0; i < resultsArray.size(); i++){

JsonElement entryElement = resultsArray.get(i);

if(entryElement instanceof JsonObject){

//remove undesired properties, corresponding to edmx

doRefactoringForEntry((JsonObject)entryElement);

}

return feedJsonObject;

}

def JsonObject doRefactoringForFeed(JsonObject feedJsonObject){

feedJsonObject.remove("count");

feedJsonObject.remove("page");

return feedJsonObject;

}

def JsonObject doRefactoringForEntry(JsonObject entryObject){

//remove undesired properties, corresponding to edmx

entryObject.remove("age");

entryObject.remove("attendance_percentage");

entryObject.remove("constituency");

entryObject.remove("debates");

entryObject.remove("education_details");

entryObject.remove("education_qualification");

entryObject.remove("elected");

entryObject.remove("gender");

entryObject.remove("house");

entryObject.remove("in_office");

entryObject.remove("last_name");

entryObject.remove("notes");

entryObject.remove("private_bills");

entryObject.remove("questions");

entryObject.remove("term_end");

entryObject.remove("term_start");

return entryObject;

}

def String transformToString(JsonObject jsonObject){

return jsonObject.toString();

}

Integration Gateway: Understanding REST data source [6]: using JSON parser

Prerequisites

Preparation

Implementing the data structure conversion using JSON parser

Overview

1. Step: Get the payload

2. Step: Parse the JSON payload

3. Step: Refactor the JSON Elements

4. Step: Transform the JSON object to String

5. Step: Set the payload

Final: Run

Full source code

Further Reading

Get Your SAP HANA Idea Incubator Badge Today!

SCN Mission - SAP HANA Quiz Challenge is now retired

Share your #HANAStory and Win