ODATA Adapter – Delta Job are taking long time to complete
Issue – The BODS job it was taking only 30 mins to complete by the time when it was go- live, and it is daily job with delta load filter condition. Later on this jobs completion was increasing by 3 mins every day and it reached to 5hrs 33 mins. Basis team raised a concern that this job is occupying job server for 5hrs, and this might crash the job server.
Analyzing issue and fix –
Our BODS source connects to SAP Cloud Platform via ODATA Adapter datastore connection. So reached out to Source Team (ABAP Team) to check on the CDS view whether it is taking long time to extract the data at source end itself. But ABAP Team confirmed that it is direct view, but this table huge records, everyday data will be appended to this source table.
Then later we started looking on the BODS side, job delta extraction was not pushed down to DB level, and it is taking entire huge records from Source to BODS layer then applying the delta logic and also due to the increase in the source every day, there is increase in job execution.
Before Optimization –
Why was it not push down?
The ODATA Adapter use API Link to connect with SAP Cloud. The date format which is passed on to the delta filter condition is “YYYYMMDD”. The date format which is passed is not in the expected format of API, so BODS couldn’t push down to DB level.
To Push down the delta extraction to DB level we need to change it to below “YYYY/MM/DDTHH:MI:SS”. After changing the date format job completion time has reduced to more than 100%. Now job completing less than 12 mins.
This is the working solution, Which I have implemented in my development. Thanks for reading my blog !!!
I will be covering more interesting topics, solution which have implemented and experienced in upcoming blogs. Please do provide your feedback which will help me in improving my inputs.
Do follow my blogs to know more on data migration and integration topics.
Thanks for your time!!!
It's very good article and useful.