Understanding of the OData service in CPI in implementing looping process

Hello everyone, hope everyone is doing well and staying safe and healthy. For the past few months, I am working on an S4 greenfield implementation. In this project, the middleware that I had suggested was SAP cloud platform integration services aka CPI.

During the realization stage of integration scenarios, I was working on a particular scenario in which I had to use an OData service that was exposed from CDS view in S4 HANA to integrate to Salesforce and some 3 rd. party tools. Initially, I taught that it was a straightforward solution until I hit a roadblock which just halted my development. In this blog, I will provide you my understanding of OData service behavior that may be useful in developing complex i-flow using CPI.

The first thing to consider, when using the OData service exposed from an S4 On-Premise system is that there is always a possibility that a Fiori app is been used to trigger the process. When we have a Fiori app in the S4 system being used then SAML2 single sign-on (SSO) will be enabled. When you try to perform a GET query using an OData adapter then it may end up in processing status for multiple days. This means in the backend CPI is unable to bypass the SAML2 and executive your query.

Note: As a default step we all use POSTMAN client big time to get the initial testing of the APIs before implementing the same in CPI. I also did the same, but the POSTMAN client will resolve SAML2 automatically and get you the data. After some investigation, I understood that SAML2 was a culprit. The CUSTOM query in OData adapter came in handy using which I could configure the SAML2 = disable and then could fetch the query. Please see the screenshot below for configuration details.

Now, let’s talk about the second issue which halted my integration development. The above fix worked when there were a certain set of records in the table. I was able to retrieve around 3k records in a single FETCH. Initially, I taught the problem was fixed and so I had asked my team to load more data to test FETCH, it broke the integration and went into the status processing. Behind the scenes, it had impacted the CPI Node which was not allowing other i-flows to work.

I reached to SAP to get this investigation underway. Initially as usual they requested access to look into the issue and they said that it is a node issue so need a restart, but the bigger problem was to resolve the FETCH issue. Here is the SAP note if you get the i-flow in message processing status and need a node restart.

https://launchpad.support.sap.com/#/notes/0002399949

I knew that the looping concept would work for a large set of data but wanted to understand how and when to adapt the looping process. I just did not want to use looping processing blind-sided without knowing when to use. I worked with SAP for around 4 weeks to get the answer and finally found one issue from my investigation and reported the same to SAP which was a nextLink when a FETCH is done using postman as shown in the screenshot below.

This was the second culprit which was blocking the integration to FETCH the data. The issue here is that the OData service is returning the records with the next link in the response as below. In CPI, it needs to be looped on the next link until all records are fetched. This behavior cannot be changed since CPI adheres to the OData specifications.

<feed ……

…….

<link rel=”next” href=”ProductPrices?sap-client=900&amp;$skiptoken=100″/>
</feed>

As per the OData spec, the next link indicates that there are more records in the server and hence we use this link to fetch the remaining records.

In this service First call returns 89 records and nextlink <link rel=”next” href=”PP?sap-client=900&amp;$skiptoken=100″>

The nextLink response is empty feed with nextlink = PP

?sap-client=900&amp;$skiptoken=200

This next call gives empty feed with nextlink = PP?sap-client=900&amp;$skiptoken=300

The above call gives empty feed with PP?sap-client=900&amp;$skiptoken=400 and so on.

The response of the next link gives an empty feed with another next link indicating more records. and this goes on infinitely. So, whenever you test the API endpoint look for nextLink in your FETCH. If so, you can make a call to go to the looping process. As of now, we have a dependency on a POSTMAN client. I hope SAP will update the OData adapter to provide more information to the developer so that a correct decision can be made in developing the scenario. The reason I put out this blog is that I could not find any answer on the community for my posted questions. So, taught it would be helpful.

Hope this helps.

Happy Blogging.