This  blog is about Integration Gateway in SAP Mobile Platform 3.0.

Since SP05 release, Integration Gateway supports REST services as data source.

You can now expose any REST service as OData service through Integration Gateway.

With this blog, I’d like to help you understand what has to be done.

And why.

This is kind of introduction, please stay tuned for the upcoming posts which will cover more details.

Prerequisites

  • SAP Mobile Platform 3.0 SP05 (SMP) installation.
  • Eclipse Kepler with SAP Mobile Platform Tools installed
    check this blog for help
  • Basic knowledge about SMP and Integration Gateway
    Check this blog for some introduction

Overview

  1. REST service as data source
  2. Create OData Implementation Project
  3. Create OData Model
  4. Bind data source
  5. Create Custom Code
  6. Implement the Custom Code
  7. Deploy
  8. Runtime

1. REST service as data source

Within the internet, there are lots of free REST based webservices that can be used for testing and demoing.

I’ve found a suitable one, which is free, doesn’t require registration and returns the payload in XML.

We’ll use it in the following example.

  Home: http://www.gisgraphy.com/

  Example REST call: http://services.gisgraphy.com/street/streetsearch?lat=52.52&lng=13.41

Try it out.

Please note:

Within the title of this blog, I’ve promised to keep this tutorial VERY simple.

We’ll do so by completely ignoring the response of the REST service.

It doesn’t need to make sense, it is only used for understanding the REST data source in Integration Gateway.

2. Create OData Implementation Project

Start Eclipse, change to the OData Perspective: Window -> Open Perspective -> Other… -> OData

Create a new OData Implementation Project: File > New > SAP Mobile Platform Odata Implementation Project

Provide an arbitrary name for  the project and the model.

Choose the “Target Runtime Server” as “SAP Mobile Platform 3.0 SP5” (should be preselected)

3. Create OData Model

After creation of the OData Implementation Project, the graphical OData modelling tool is opened.

If not, open it with a double-click on the <your_name>.odata file

Create an EntityType and rename it to “Street”

Create one additional property with name “StreetName”

Note:

our OData model has nothing to do with the REST service, as we’re going to ignore the REST response.

At this point, we have all information that we’ll need later:

  • REST service URL
  • EntityType name
  • EntitySet name
  • Property names

4. Bind data source

Within the “Project Explorer”, select the <your_name>.odatasrv file, open the context menu and choose “Select Data Source”.

We have only one EntitySet, so it is already selected.

We’re going to implement the QUERY operation, so this has to be selected (if not already preselected)

Mark the checkbox for “REST Service”

Press “Next”.

Now we have to enter the “relative request URL”.

At this point, we have to understand:

We have a full URL for a service that returns data: http://services.gisgraphy.com/street/streetsearch?lat=52.52&lng=13.41

We split the full URL into 2 segments:

  1. The host URL
    http://services.gisgraphy.com
    -> it will be specified in SMP as destination for the (future) OData service
    -> it doesn’t have trailing slash.
  2. The relative URL
    /street/streetsearch?lat=52.52&lng=13.41
    -> it is entered in this wizard page.
    -> it is required to have the leading slash.

Now paste the relative URL into the wizard and press “Finish”

5. Create Custom Code

Within the “Project Explorer”, expand the <your_name>.odatasrv node and open the context menu on the “Query” node

Choose “Define Custom Code” and in the subsequent popup, select “Groovy”

Press “Finish”, the script file is created and is opened in an editor.

I recommend to follow the few steps described here in order to improve the development experience with Groovy.

6. Implement the Custom Code

This is finally the main section of this blog.

What are we required to do here?

We have 2 callback methods:

processRequestData()

    ->We ignore it in this blog.

processResponseData()

    ->We focus on this one.

We have to understand:

When a user invokes the (future) OData service on SMP, the request goes to SMP and to the Integration Gateway component.

Integration Gateway knows to which data source to connect in order to fulfill the request.

The data source returns the desired data.

Now Integration Gateway has to take this structured data and re-structure it in the proper OData structure, before returning it to the user.

And this is the pain point in the case of REST data source:

it is structured, but HOW?

There’s no spec on which Integration Gateway can rely.

And this is why we, the implementers of the OData project, come into the game:

We are supposed to understand the structure of data of the REST service that we want to use.

We are supposed to help Integration Gateway to convert the structure of the REST service into the structure of our OData service.

We are supposed to implement this in the custom code.

The method “processResponseData() is invoked by the Integration Gateway Framework right after it has received the response from the REST service.

In this method, we have to

1) retrieve the response body of the REST service.

2) convert it

3) return it

Let’s have a closer look

6.1 The REST response

The method processResponseData() has a parameter called “message” which is of type com.sap.gateway.ip.core.customdev.util.Message

This object provides access to the REST response via the following statement:

String restResponse = message.getBody().toString();

6.2 Conversion

Let’s be optimistic for now and assume that the call to the REST service has been successful.

In this case, we get a long string which contains the response that is returned if we call the REST service directly:

http://services.gisgraphy.com/street/streetsearch?lat=52.52&lng=13.41

rest_orig_payload.JPG

In our script, we have to manipulate this string in order to convert it to the desired structure.

But I’ve promised to be very simple, so let’s just ignore the REST response in this tutorial.

Let’s focus only on returning the correct structure.

Let’s do a hack that is allowed for educational purpose;-)

Let’s hardcode the return-structure.

6.3 The return structure

To be short:

Integration Gateway doesn’t require from us that we do the full conversion of the REST-response to an OData-response.

Integration Gateway will provide the correct OData format that complies with the spec.

What we have to do is:

-> provide the data in the structure that Integration Gateway expects.

I really mean EXACTLY in the structure that Integration Gateway expects.

Don’t worry, it is easy:

EntitySet

    Entity

          Property

Since we are in a QUERY operation, we have to provide the correct entity set name.

Furthermore, a QUERY returns a list, so there can be one or more (or zero) entity types:

EntitySet

    Entity1

          Property

          Property

          Property

    Entity2

          Property

          Property

          Property

    …

Even more precise, see here the expected structure in the expected format.

The format is in our example xml, because the the chosen REST service returns xml.

BTW, it has to be valid xml, with values and closing tags (otherwise we get an error at runtime)

<EntitySetName>

    <EntityName1>

          <PropertyName1>“value of property1”</PropertyName1>

          <PropertyName2>“value of property2”</PropertyName2>

          <PropertyName3>“value of property3”</PropertyName3>

    </EntityName1>

    <EntityName2>

          <PropertyName1>“value of property1”</PropertyName1>

        <PropertyName2>“value of property2”</PropertyName2>

          <PropertyName3>“value of property3”</PropertyName3>

    </EntityName2>

</EntitySetName>

Up to here it has been the generic description.

Now let’s pllay it to our concrete example.

Look at the OData model that you’ve created in the graphical editor:

See it?

These are the names which you have to provide in the return structure.

I recommend using copy&paste in order to be sure to avoid tedious errors at runtime.

Now let’s build the response string for our example.

We copy the names from our odata model and paste them into our script file.

The expected structure would look like this:

<StreetSet>

    <Street>

          <StreetID>1234567</StreetID>

          <StreetName>MainStreet</StreetName>

    </Street>

</StreetSet>

Note:

I don’t need to mention, that the names of the properties etc have to be correct.

I mean, really correct, including the right case.

Now let’s add the following code to our Groovy script:

message.setBody(

            “<StreetSet>”+

                    “<Street>”+

                          “<StreetID>1234567</StreetID>”+

                          “<StreetName>MainStreet</StreetName>”+

                    “</Street>”+

            “</StreetSet>”);

This is our hardcoded payload in the expected structure, according to our OData model.

We still need to specify the format of the payload in the respective header.

message.setHeader(“Content-Type”, new String(“xml”));

and don’t forget to add the import statements:

import com.sap.gateway.ip.core.customdev.logging.LogMessage

import com.sap.gateway.ip.core.customdev.util.Message

We’re done.

For your reference, here’s the full (dummy) implementation of our mehtod:


def Message processResponseData(message) {
    String restResponse = message.getBody().toString();
    message.setBody(
        "<StreetSet>"+
            "<Street>"+
                "<StreetID>1234567</StreetID>"+
                "<StreetName>MainStreet</StreetName>"+
            "</Street>"+
        "</StreetSet>");
    message.setHeader("Content-Type", new String("xml"));
    return message;
}








Don’t forget to save the file.

7. Deploy

Within the Project Explorer, select your project and from the context menu choose “Generate and Deploy Integration Content”.

You may specify some service details as desired and press “OK”.

After you get the success popup,

you can proceed and have a look in the Gateway Management Cockpit on the SMP server

Of course, here we assume that the SMP is configured in your Eclipse.

If not, you have to do so.

Go to Window -> Preferences -> SAP Mobile Platform Tools -> Server

And specify the details:

8. Runtime

Open the Gateway Management Cockpit at e.g. https://localhost:8083/gateway/cockpit

You might need to press “Refresh” to see your service.

Before invoking it, we need to configure the destination.

As we’ve learned above, we’ve split the full URL of our REST service into 2 segments.

The second segment is kept in the service implementation, that’s what we’ve done above.

The first segment has to be configured as destination in our SMP.

Let’s quickly do that.

In the Gateway Management Cockpit, press the “Destinations” tab and click on “New Destination”.

Provide an arbitrary name for the destination.

The type is HTTP

The URL is the host URL that we’ve split above: http://services.gisgraphy.com

Authentication is not required for the REST service that we’ve chosen.

Note:

if you’re a keyboard user, you might get crazy because e.g <arrow left> will move the cursor to the previous input field, instead of moving the cursor inside the field to the previous character.

In case you didn’t know: you have to press F2 to edit the content of the current input field.

Yes, sir!

Save the destination.

Result should be a success popup.

You can now select the new destination  in the table, then press “Test Connection”.

Note:

The “Connection Test” is not 100% reliable, in the sense that it might report that the connection test fails, even though the connection is OK.

As we’ve seen above, for the REST data source, we have to split the URL. It can be the case that the first segment, the host, doesn’t respond as desired to the “Connection Test”, but it works fine for the full URL at runtime.

Note:

Depending on your entwork, it might be necessary to add proxy settings to your SMP server.

This is done at

https://<your_server>:8083/Admin/

In the “Settings”-tab and “System” sub-tab

Next, you have to assign this destination to your service.

Go back to the “Services” tab, click on your service in order to go to the details screen.

Click the “Add Destination” button and assign your new destination to your service:

Press OK.

Now you can click the hyperlink in the section “Service Access”:

The service document is opened.

Here you can see the one EntitySet that we’ve specified in our OData model:

This step should really always work fine.

Because until now we haven’t dealt with any backend-data-source-connection.

Next step is to invoke the EntitySet at:

https://localhost:8083/gateway/odata/SAP/REST_PROJECT_VERY_SIMPLE;v=1/StreetSet

(as usual, you have to adapt the sample URL to your SMP-installation)

The result is our hardcoded payload:

That’s it.

We’ve understood how the REST data source has to be implemented.

We’re now prepared for real implementation – and real headaches…;-)

BTW: I’ve attached the OData model and the custom script file for your convenience.

To report this post you need to login first.

20 Comments

You must be Logged on to comment or reply to a post.

  1. Anonymous

    Dear Carlos,

    Thanks for sharing this .

    I have successfully deployed the service on SMP gateway and I have got the service document :

    <workspace>

    <atom:title>Default</atom:title>

    <collection href=”StreetSet>

    <atom:title>StreetSet</atom:title>

    </collection>

    </workspace>

    and the metadata

    <EntityType Name=”Street>

    <Key>

    <PropertyRef Name=”StreetID/>

    </Key>

    <Property Name=”StreetID Type=”Edm.String Nullable=”false/>

    <Property Name=”StreetName Type=”Edm.String/>

    </EntityType>

    <EntityContainer Name=”default m:IsDefaultEntityContainer=”true>

    <EntitySet Name=”StreetSet EntityType=”verySimple.Street/>

    </EntityContainer>

    </Schema>



    but when trying to get the data itself using this url :


    https//localhost:8083/gateway/odata/SAP/VERYSIMPLE_SERV;v=1/StreetSet


    it gives me:


    Error Message : Could not perform the operation, contact your System Administrator. Check the log file for details : Transaction ID : “”


    from admin log :

    readEntitySet failed: entity set = StreetSet, message =: org.apache.olingo.odata2.api.exception.ODataApplicationException: : TransactionID =  “”



    could you please help me on this ?


    I’m using SMP 3.0 SP06 , Is there any changes from SP05 to SP06 may case this error, as I’m also tried to integrate OData services and using Integration gateway and I managed to get service document and metadata, but when I come to execute the entity set it gives me :


    Could not perform the operation, contact your System Administrator. Check the log file for details : Transaction ID :while trying to invoke the method com.sap.gateway.core.api.srvrepo.IServiceInfo.getNamespace() of a null object loaded from local variable ‘service’



    Thanks

    Hossam

    (0) 
    1. Carlos Roggan Post author

      Thanks, Hossam;-)

      I’ll check this on SP06

      In the meantime, please have a look into the server log file, as there might be more useful information.

      What I’m also doing is to start the server in command shell, as there’s sometimes useful output as well.

      Since log output is large, can you contact me via msg?

      Cheers,

      Carlos

      (0) 
    2. Carlos Roggan Post author

      Hi Hossam,

      I’ve tested the scenario on a SP06 server and it works out of the box.

      What could be the problem on your side?

      I would guess that there are 2 possible error causes:

      1. The error occurs only when calling the entity set.

      So only now, the real call to the REST service (gisgraphy) is being done.

      For displaying the service doc and the metadata doc, no call to gisgraphy is necessary.

      So this is why I would assume that there’s something wrong with your destination.

      Have you done the “Test Connection” ?

      Is it green?

      Have you defined the “relative URL” correct in Eclipse?

      2. The second reason could be that there’s a typo in the properties.

      So when processing the response, the IGW framework takes the payload, parses it and assigns the values of the properties in the payload to the properties that are in the odatamodel. If a property name doesn’t match, then the server throws an exception

      Hope that this helps!

      Cheers,

      Carlos

      (0) 
      1. Hossam Abdelazeem

        Dear Carlos,

        Thanks a lot for your response.

        here are my answers for your questions:

        1. Yes, I have tested it .. you can check these screenshots for more details:

        1.PNG

        2.PNG

        3.PNG

        4.PNG

        9.PNG

        10.PNG

        my script:

        7.PNG5.PNG

        when calling the service :

        8.PNG

        admin log after calling will have only two errors :

        6.PNG

        what do you think 🙂 ?

        (0) 
        1. Carlos Roggan Post author

          Hi Hossam,

          Your screenshots have clarified some questions.

          What else could we check.

          1) One idea could be:

          Are you using an Eclipse-IDE that matches the new SP6 server?

          When creating an SMP project, you have the choice to select the “Target Runtime”

          This is the SMP where you’re deploying to.

          I think that based on this selection, the matching dependencies are generated into the
          Manifest.MF file.

          These dependencies are marked as “optional”, such that the bundle will still be
          resolved after deployment.

          So you would get an error only once the dependencies are used.

          So when creating the project you have to choose “Target Runtime” as SP6

          2) Regarding the log messages:

          We need more info than what’s visible in the screenshot.

          https://localhost:8083/Admin/ -> Logs -> LogFiles

          Open the smp-server.log

          Scroll down to the very end and check the messages

          3) I’ve seen that you’ve added tracing to your script.

          That’s good.

          You would now have to check in the log if this output is written.

          Cheers,

          Carlos

          (0) 
          1. Hossam Abdelazeem

            1) I already using SP06 in eclipse

            2) log contain this error :

            |

            2015 03 12 18:12:48#+00#ERROR#com.sap.gateway.core.ip.provider.data.CamelDataProvider##anonymous#http-bio-8083-exec-51###[Gateway][TECHNICAL][ODataServiceProvisioningFailed]:Failed to Provision Odata service. For input string: “” |

            2015 03 12 18:12:48#+00#ERROR#com.sap.gateway.core.ip.provider.data.CamelDataProvider##anonymous#http-bio-8083-exec-51###[Gateway][BUSINESS][ReadEntitySetFailed]:Data Provider – readEntitySet failed: service = VERYSIMPLE_SERV ,  entity set = StreetSet , message =  java.lang.NumberFormatException: For input string: “” |

            2015 03 12 18:12:49#+00#ERROR#com.sap.gateway.core.odata4sap.ODataErrorCallbackImpl##anonymous#http-bio-8083-exec-51###handleError(): failed to serve request for


            URI https://servername:8083/gateway/odata/SAP/VERYSIMPLE_SERV;v=1/StreetSet


            message = Error Message :: Transaction ID : “” |

            3) error happen before even my custom script called

            (0) 
            1. Carlos Roggan Post author

              Hi Hossam,

              I remember I had such error message in the past: For input String “”

              The reason was that proxy setting was missing.

              Can you try entering proxy settings, please?

              After saving the page, please restart the SMP server in order to ensure that the new proxy settings are propagated to Integration Gateway.
              Cheers,

              Carlos

              (0) 
              1. Hossam Abdelazeem

                why should I enable proxy ?

                my SMP server is exposed on azure cloud and I can access the data source using url directly.

                12.PNG

                this screenshot is from SMP server

                and by the way ,

                the server don’t have any proxy setting.

                13.PNG

                (0) 
                1. Chao Zheng

                  Hi Hossam,

                  Have you resolved this issue? Unfortunately I encountered exactly the same issue with you, so please help to share if you have any advanced experience, many thanks!

                  (0) 
                  1. Hossam Abdelazeem

                    Hi Chao Zheng


                    Unfortunately I didn’t resolve it, I was using REST API to access MS SQL Database, as I thought that Integration gateway can’t connect to MS SQL but I was wrong, I discovered that Integration gateway can communicate to MS SQL Server using JDBC connection, also I thought that you can’t communicate to REST service except through integration gateway and I was wrong too , you can add REST service as additional endpoint and in this case you can execute it directly and get JSON results directly on client but in this case you will loss a lot of useful features in OData client SDK like offline store and cashing.

                    (0) 
  2. Sri Patlu

    Carlos ~ Thanks for creating this blog on exposing REST service as oData service. Using your instructions I was able to create a test project that exposes REST service as oData service, and deploy it to our SMP 3.6 system. I configured the destination on SMP as instructed in your blog and I am able to access the service document

    I get an error message when I invoke the entity set using link https://kdwsapmbd01.<mycompany&gt;.com:8083/gateway/odata/SAP/MYTESTSERVICE;v=1/StreetSet.

    Error details is pasted below.

    <error xmlns=”http://schemas.microsoft.com/ado/2007/08/dataservices/metadata>

    <code/>

    <message xml:lang=”en-US>Requested entity could not be found.</message>

    </error>

    Portion of stacktrace is pasted below as well:

    org.apache.olingo.odata2.api.exception.ODataNotFoundException: Requested entity could not be found.

      at com.sap.gateway.core.ip.provider.model.CamelModelProvider.getSchemas(CamelModelProvider.java:106)

      at com.sap.gateway.core.api.provider.model.AbstractModelProvider.getEntityContainerInfo(AbstractModelProvider.java:89)

      at com.sap.gateway.core.api.provider.model.CacheModelProvider.getEntityContainerInfo(CacheModelProvider.java:101)

      at org.apache.olingo.odata2.core.edm.provider.EdmImplProv.createEntityContainer(EdmImplProv.java:59)

      at org.apache.olingo.odata2.core.edm.EdmImpl.getEntityContainer(EdmImpl.java:69)

      at org.apache.olingo.odata2.core.edm.EdmImpl.getDefaultEntityContainer(EdmImpl.java:184)

      at org.apache.olingo.odata2.core.uri.UriParserImpl.handleNormalInitialSegment(UriParserImpl.java:174)

      at org.apache.olingo.odata2.core.uri.UriParserImpl.handleResourcePath(UriParserImpl.java:158)

      at org.apache.olingo.odata2.core.uri.UriParserImpl.parse(UriParserImpl.java:112)

      at org.apache.olingo.odata2.core.ODataRequestHandler.handle(ODataRequestHandler.java:107)

      at org.apache.olingo.odata2.core.rest.ODataSubLocator.handle(ODataSubLocator.java:155)

      at org.apache.olingo.odata2.core.rest.ODataSubLocator.handleGet(ODataSubLocator.java:56)

      at sun.reflect.GeneratedMethodAccessor276.invoke(Unknown Source)

      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

      at java.lang.reflect.Method.invoke(Method.java:606)

    Appreciate if you can take a look at the error and advise what I may be missing / doing wrong in my project?

    (0) 
    1. Carlos Roggan Post author

      Hello Sri,

      From the error message it is hard to assume the reason of the failure.

      The message says that the ‘entity’ could not be found.

      The stack trace shows that the FWK, before fetching the backend data, is reading the metadata:

      The EntityContainer, the Schema, then the Entity, and here it fails.

      So it seems that anything is wrong with your odata model.

      Have you tried opening the $metadata document?

      I guess this should fail as well (and with the same error message)

      Could you please check your odata model?

      (0) 
      1. Sri Patlu

        Carlos ~ Thanks for looking into the issue. I’m able to open the $metadata document and its contents are pasted below. I have checked odata model and the project overall and all seems to be in order. If possible, can you attach your working project so I can repurpose it to our SMP as see if that yields a different result? Thanks!

        <edmx:Edmx xmlns:edmx=”http://schemas.microsoft.com/ado/2007/06/edmx Version=”1.0>

        <edmx:DataServices xmlns:m=”http://schemas.microsoft.com/ado/2007/08/dataservices/metadata xmlns:sap=”http://www.sap.com/Protocols/SAPData xmlns:ux=”http://www.sap.com/Protocols/OData4SAP/UX xmlns:gp=”http://www.sap.com/Protocols/SAPData/GenericPlayerm:DataServiceVersion=”1.0>

        <Schema xmlns=”http://schemas.microsoft.com/ado/2008/09/edm Namespace=”MyModel>

        <EntityType Name=”Steet>

        <Key>

        <PropertyRef Name=”SteetID/>

        </Key>

        <Property Name=”SteetID Type=”Edm.String Nullable=”false/>

        <Property Name=”StreetName Type=”Edm.String/>

        </EntityType>

        <EntityContainer Name=”default m:IsDefaultEntityContainer=”true>

        <EntitySet Name=”SteetSet EntityType=”MyModel.Steet/>

        </EntityContainer>

        </Schema>

        </edmx:DataServices>

        </edmx:Edmx>

        (0) 
        1. Carlos Roggan Post author

          Hello Sri,

          in your edmx, I can see:

          <EntitySet Name=”SteetSet EntityType=”MyModel.Steet/>

          but in the above mentioned URL, you’re invoking:

          odata/SAP/MYTESTSERVICE;v=1/StreetSet

          This should be the reason…;-)

          Cheers,

          Carlos

          (0) 
          1. Sri Patlu

            I remember checking the URL several times. URL works with correct entityset name. Sincerely appreciate your efforts to resolve the issue.

            Sri

            (0) 
  3. Amit Nalawade

    While adding a destination from gateway cockpit for a service, i always get a error message “Service Unavaliable”. Any hints on why this may be happening? Does this mean SMP is unable to reach the destination?

    Thanks!

    (0) 
    1. Carlos Roggan Post author

      Hi Amit,

      I think that it means that a 500 was returned for the invoked URL

      Maybe you should file a question at the discussion?
      Please provide more details, have you done “TestConnection”, what kind of Backend, what kind of service, proxy required/entered?

      Cheers,

      Carlos

      (0) 
      1. Amit Nalawade

        I am not able to add destination at all, so there is no way i can test the connection from the cockpit. It fails and gives the error message while trying to add a destination.

        (0) 

Leave a Reply