Skip to Content

This blog is part of a collection of blog entries that shows architectural concepts and configuration of the SAP PI REST Adapter. We also added some sample scenarios to make it easier for you to understand how your scenario can be implemented using the PI REST Adapter.

If you haven’t done so far, best is to start with the very first blog PI Rest Adapter – Don’t be afraid within the blog series covering the concepts of the REST adapter. A complete list of all blog entries can be accessed from here PI REST Adapter – Blog Overview.

The current blog describes how to poll a REST API. The sender REST adapter in polling mode has been supported since release 7.31 SP16 / 7.4 SP11.

Scenario

We would like to frequently poll video information from Google’s Youtube REST API, and store the same on a file system. Furthermore, the resultset should be split into individual messages per video ID, and already polled results should be discarded.

In the SAP Process Integration Designer perspective of the NetWeaver Developer Studio (NWDS), I have defined an Integration Flow with a REST sender channel and a file receiver adapter.

01 iflow.png

Let’s focus on the configuration of the sender channel.

Configuring the REST sender channel

Double-click on the sender adapter to open the channel editor. Select the REST adapter type, and from the Message Protocol drop down menu the entry REST Polling.

02 Adapter type.png

Option 1: Incremental request based on timestamp of last call

In order to better understand the configuration below, let’s take a look at a sample response in JSON format. The REST API returns an array of items each having a unique ID stored in the etag field.

05 sample json response.png

First switch to sub-tab Data Format below tab Adapter-Specific.

The format is JSON, so select JSON from the Data type drop down menu. We would like to convert the JSON into XML, so select the Convert JSON to XML check box, and add a wrapper element to ensure that the converted XML format contains one root element only.

As mentioned below, the message should be split into individual messages per video ID. Select Split Result into Multiple Messages check box. As Array Containing Messages maintain items (See message format above).

Furthermore, duplicates should be removed. Select the Filter out Duplicates check box, and maintain etag as Unique ID Element (See message format above).

To place incremental requests, we would like to use the timestamp of the latest call. The value is stored between the calls and can be used in the REST URL as a placeholder with name incrementalToken. By the way, alternatively you can also use an XPath expression or a JSON element from the response of the last call such as next Page indicator or similar (shown below). From the Incremental Type drop down select entry Timestamp of Last Call. The timestamp format of the API complies with ISO 8601, and needs to be specified as follows: yyyy-MM-ddTHH:mm:ssZ with T and Z being constants. The REST adapter follows the Joda time format, see DateTimeFormat (Joda time 2.2 API). So, we need to place the constants in quotes. Maintain the Timestamp Format attribute as yyyy-MM-dd’T’HH:mm:ss’Z’, and define an initial value that complies with the format.

04 data format.png

Switch to tab HTTP Request, and maintain the Target URL as follows: https://www.googleapis.com/youtube/v3/search?key=<your API key>&part=id&q=SAP&maxResults=50&order=date&publishedAfter={incrementalToken}. Note, I added the placeholder incrementalToken in curly brackets which holds the timestamp of the last call. As HTTP Operation select GET. Finally, define a polling interval, here 3600 seconds.

03 http request.png

Option 2: Incremental request based on response content

As another option I would like to show you how to configure the incremental request in case that many pages need to be requested. As you can see below, in the response two additional elements are added holding the previous (prevPageToken) and the next page token (nextPageToken). Latter can be used in the URL to gather the next response page.

01 Chrome.png

From the Incremental Type drop down select the entry Response Content, and maintain the incremental token element as nextPageToken. This will store the value of the nextPageToken element into the placeholder incrementalToken.

From release 7.31 SP17 / 7.4 SP12 / 7.5 SP01 onwards, you can define an action in case that the very last page has been reached and hence the nextPageToken element is missing. You have the following options:

  • treat as error which is the default
  • skip current poll and retry later
  • use an empty token value
  • use the initial token value
  • use a custom token value

Here, we chose the option Use Empty Value for Token.

02 Config.png

On tab HTTP Request, maintain the Target URL as follows: https://www.googleapis.com/youtube/v3/search?key=<your API key>&part=id&q=SAP&maxResults=5&pageToken={incrementalToken}. Note, I added the placeholder incrementalToken in curly brackets which holds the token of the next page.

03 Config.png

I hope this blog was helpful to understand how to configure the REST sender channel in polling mode. If you like to learn more, check out the other blogs in the series, accessible from the main blog PI REST Adapter – Blog Overview.

To report this post you need to login first.

25 Comments

You must be Logged on to comment or reply to a post.

      1. Sateesh Nagabhiru

        Hi Alexander,

        I did same way on earlier also.

        and I get this error

        Error while processing inbound message. java.lang.RuntimeException: java.net.URISyntaxException: Illegal character in query at index 83: https://tools-dev:9307/ceres/data/basic?lastChangedTime={incrementalToken}: java.net.URISyntaxException: Illegal character in query at index 83: https://tools-dev:9307/ceres/data/basic?lastChangedTime={incrementalToken}: Illegal character in query at index 83: https://tools-dev:9307/ceres/data/basic?lastChangedTime={incrementalToken}


        Thanks & Regards,

        Sateesh

        (0) 
      2. Sateesh Nagabhiru

        Hi Alex,

        Our SP16 REST adapter features is supporting to convert from JSON to  XML with Multiple data sets?

        I heard from some threads those SP14 & SP15 are not supporting multiple datasets when converting from JSON to XML

        if this is also same case on SP16 , then how can I  proceed to accepts multiple Datasets.

        Please suggest me on this.

        Thanks & Regards,
        Sateesh N

        (0) 
        1. Alexander Bundschuh Post author

          Hi Sateesh,

          the REST adapter does support JSON arrays for both JSON to XML and XML to JSON conversion. There is only one restriction when an XML element which is unbounded contains only one occurence, it won’t add the brackets [ and ] to the JSON, if there are at least two entries it will. Some REST APIs need the arrays even if the element occurs only once. This will be addressed with SP17.

          Alex

          (0) 
  1. Sateesh Nagabhiru

    Hello Alex,

    I am using this URL to call REST webserive using Rest Sender poll adatper

    url: https://tools-dev:9307/ceres/data/basic?lastChangedTime={incrementalToken}


    In the Incremental Request tab


    IncrementalRequest.jpg

    If I used Time stamp format  is this way yyyy-MM-dd’T’HH:mm:ss’%2B02:00′ then only PI retrieved latest changes from REST service.

    If I used Time stamp format is this way yyyy-MM-dd’T’HH:mm:ss’Z’, PI doesn’t receive changes from REST webservice

    Currently CET time is 2 hours ahead into UTC time, after october this year again CET will be changed and at that only 1 hour ahead into UTC time. If that is the case I need to change a communication channel twice a year.

    My complain is I do not want change the communication channel twice a year, can you please give me some idea, how I can put permanent Time Stamp format in the Incremental Requests?

    Please suggest me.

    Thank you

    Sateesh

    (0) 
  2. Pradeep Mysore

    Hi Alex,

    Polling is a great feature within REST Sender adapter. However there appears some issue with HTTP header parameters. Though we have set the parameter. REST sender adapter is not passing that to the remote system. We are trying to pass some system specific information and the call is failing because the header parameters is not passed.

    Is this is a known issue? Are there any workaround or any patch available. We are currently on the latest SP 11.

    I appreciate any response from you.

    Thanks

    -Pradeep

    (0) 
    1. Ashutosh R

      Hi Pradeep,

      did your problem got solved, as we are also facing issue with rest sender polling  with adding custom header to be passed with request message.

      regards,

      ashutosh

      (0) 
      1. Alexander Bundschuh Post author

        Hi Ashutosh, Pradeep,

        this was a bug and should be solved with the latest patch, if you are on the latest patch and still face the issue, please raise an incident ticket

        Alex

        (0) 
  3. Pradeep Mysore

    Thanks Alex,

    It is was a bug and i had opened a incident ticket and it is resolved in following note.

    29.10.2015 – 10:34:30 EST – Reply by SAP

    Hello,
    The issue is corrected for 7.40 SP11 with SAP Note 2202836 #REST
    Poller channel ignores custom headers#

    We did apply this note and it has indeed fixed the issue.

    -Pradeep

    (0) 
  4. RAVIJEET DAS

    I had below queries on the REST Polling adapter :

    1. How do you view the runtime target HTTP URL, do you see in audit logs, I didn’t find that. So if I use {IncrementalToken} with Last datetimestamp, I would like to see what value gets replaced in the token each time the interface runs. If the call errors out I can see the error URL but not when it is a success I can’t see the URL

    2. In ” Timestamp of last call ” there is a field for Initial value, when does this get into effect. The first time we run this interface ? Say I am stopping this channel and again restarting it, then what time it takes for ” Timestamp of last call ” or does it take the initial value which I mention in configuration?

    3. There is an option of MessageSpilt – Split Result into Multiple Messages, here I can specify the xpath of the element or table which iterates. So how does this spilt happen, does it slit say I have 5000 records into batches of 50 or it is 5000 single records ?

    (0) 
  5. RAVIJEET DAS

    Hi Alex,

    Is there any custom way of handling below HTTP polling scenario. I am trying to get candidates from a Recruiting system and I need to pass below values in URL:

    https://api.jobvite.com/api/v2/candidate?api=<key>&sc=<secret>&format=json &start=1&count=50&datestart=2014-01-01&dateend=2014-07-01

    1. So my variables are “start” which would the next starting index i.e. 1…51…101

    2. datestart would be the last time I ran this interface

    3. dateend would be current time.

    I tried the incrementalcount approach but how do I handle current date also how do I handle both incremental date and nextPageToken ?

    Thanks in advance

    Ravijeet

    (0) 
    1. RAVIJEET DAS

      Hi Alex,

      I was trying to check how do we implement the below scenario using incremental token:

      If we are iterating the records page by page, say we have 2000 records and max we can get in one request is 500, the 4 request call will be as below:

      https://api.jobvite.com/api/v2/candidate?api=api_key&sc=abc&start=1&count=500

      https://api.jobvite.com/api/v2/candidate?api=api_key&amp;sc=abc&start=501&count=500

      https://api.jobvite.com/api/v2/candidate?api=api_key&amp;sc=abc&start=1001&count=500

      https://api.jobvite.com/api/v2/candidate?api=api_key&amp;sc=abc&start=1501&count=500

      What value do we store in nextPageToken ?

      Thx in advance

      Ravijeet

      (0) 
  6. Fabian Engel

    Hello,

    we have PO 7.4 SP12 and the Rest Polling Protocol is not able to choose in the Communication Channel.

    Do you have any Idea?

    Thanks and regards,

    Fabian

    (0) 
  7. Filipe FERREIRA

    Hi,

    We have the same issue as Fabian with same PI version 7.40 and SP12. The ‘REST Polling’ Message Protocol is missing in the sender adapter.

    Does anybody know if this feature is still supported with 7.40 SP12?

    Thanks,

    Filipe

    (0) 
  8. Frederick-Claud Dimmer

    Hi,

    I want to poll yesterdays performance data from our PO using the performancedataqueryservlet. I’m passing in the central adapter engine as component. In addition to that I need to specify a begin and end parameter. The request URL looks as follows:

    http://<host&gt;:<j2eeport>/mdt/performancedataqueryservlet?component=<component>&begin=2016-10-17%2002:00:00.0&end=2016-10-18%2002:00:00.0

    I wanted to use the incremental timestamp but only one timestamp is supported. Is there any solution to this problem?

    (0) 
    1. Rafael Vieira

      Hi Frederick,

      Did you get an answer for this question?
      I’m currently setting a REST sender which has to poll an API and I need to use Begin Date and End Date in its URL.

      Can you share your findings, please?

      Tks.

      (0) 
  9. Rafael Vieira

    Hi,

    I need to call an URL as follow:
    http://<host>/rest/<service>/getVehiclesbyDateRange?key=%5Bhard_coded_key%5D&amp;beginDate=2016-12-09&endDate=2016-12-10

    Begin and End date must be the same so the service would return the list of vehicles for that particular day, only.
    But Begin/End Date must be automatically incremented in each day, always to get the current day’s vehicle list.

    Can anyone suggest how to achieve this in the REST Sender with poll?

    Tks!

    (0) 
  10. Dheeraj Kumar

    Hello,

    Please help to resolve below issue.

    Scenario = REST Webservice –> PO –> ECC

    I have configured Sender REST Polling Adapter. Every 30 minutes this adapter starts polling sales order from REST Service. I have selected parameter “Filter out duplicates”. So every time it polls it picks up only unique sales order. From morning it works fine but suddenly at 4:30 PM poll time it picks up all the sales order irrespective of duplicates.

    Please help to resolve this issue as this is a production issue.

    Regards,

    Dheeraj kumar

    (0) 

Leave a Reply