You can perform batch operations in the OData V2 Adapter for SAP Cloud Platform Integration. Batch, as its name suggests, implies that multiple requests are clubbed into one HTTP call. What this means is that if you are updating 100 records, instead of making 100 PUT or update request, the adapter would just make one call to the server with all the essential data. This would greatly improve time taken to achieve such requirements, also improving the performance.
The current OData V2 Adapter in SAP Cloud Platform Integration supports POST, PUT, MERGE,GET and DELETE HTTP operations in batch. Using the POST operation, you can insert ‘n’ number of new entities in your OData backend, using PUT/MERGE operation you can update entities and DELETE operation to delete entities. We will visit POST and PUT examples in the following sections.
To know more on how the OData Service supports batch you can go through OData documentation, http://www.odata.org/documentation/odata-version-2-0/batch-processing/ which will give you a brief overview on structure and format of a batch request.
The request body of batch contains an ordered sequence of operations to be performed. These modifying operations are called ‘ChangeSet’. ChangeSets are atomic units that contain the entity on which create, update or delete operations are to be performed. The ChangeSets can also contain an unordered sequence of operations. All changes in a ChangeSet must be all processed successfully or none of them. Implementation of ChangeSet varies with different services.
Here’s an example of the request body in SAP Cloud Platform Integration for ‘Batch Request’ which is denoted by a <batchParts> tag. This contains 2 ‘ChangeSets’ denoted by <batchChangeSet> tags. It is mandatory to have at least one ChangeSet in a batch request payload.
The <method> tag denotes which operation is to be performed. Iin this case, the operation is POST and the entity set is Categories. These tags are mandatory to be sent in all batch operations. There are a few optional tags like <headers> to send custom headers to ChangeSets. The <uri> tag is used to mention the exact entity on which you wish to perform the operation. It is mandatory in case of DELETE operation. Remember to add the key and its value when using <uri> tag for DELETE, MERGE or UPDATE operations.
We are using the OData Demo V2 service, http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc in this example.
Note: Soon after modelling the operation, XSD file is generated and a message is displayed based on the fields selected by you in Step 3. This is to help you to map the request payload when using message mapping.
Here’s an example of how the batch response looks like.
The first ChangeSet is the generic format and the second ChangeSet is an actual response received for the above request. Each individual ChangeSet response can be viewed under <batchChangeSetResponse> tag, which is followed by the <headers> tag. This <headers> tag denotes response headers from the OData Endpoint.
The <statusCode> and <statusInfo> gives us the HTTP status code and status line details respectively. <body> tag shows us the successfully inserted response in case of insert operations. It is empty for update operations. An error response is shown if either of the operations is unsuccessful. However, in the SAP Cloud Platform Integration’s message processing log (MPL), you will see that the status is COMPLETED. This is because, the batch requested has been accepted successfully by the OData server and it has provided a response.
The timer is configured to run once; the content modifier contains the request body. The response is captured in SFTP Server.
An example of batch request body is as seen below; we are inserting 2 new Categories with ID ‘25’ and ‘26’.
The output as observed in SFTP server is
SOAP adapter triggers the call to the OData Endpoint. The response is sent to SFTP server via SFTP adapter. The target mapping used is the XSD generated from the OData Receiver.
Batch request body is shown below. The first entity should be successfully updated and the second one should fail because there is no Category(123) currently in the OData server.
The output as observed in SFTP server is
The request payload that we will be using for this scenario is as seen below.
Notice that, the request payload contains two ordered ChangeSets(batchChangeSet).
The first ChangeSet consists of two request to be performed(batchChangeSetPart), i.e creation of ID 16 and 17. If either of the 'batchChangeSetPart' fails neither will be completed. The rollback is performed by the OData Server if failure occurs.
These 'batchChangeSetPart' are unordered, meaning that any batchChangeSetPart can be performed first irrespective of the sequence in the payload.
The second ChangeSet consists of two update request of ID 16 and 17.
Successful response for the above request payload can be seen below. As of 2.27.0 version of Cloud Platform Integration, we allow only one response to be listed in the payload for each ChangeSet.
Batch GET request requires a request payload like other Batch operations but the XML structure offers. Below is an example of a request.
In this example, we are performing 4 different GET operations, i.e. reading the first 'Product', getting all the 'Suppliers', reading 'Categories' with for ID 0 and performing function import 'GetProductsByRating' respectively. <uri> tag is necessary for Batch GET. You can also see the XML tag is <batchQueryPart> instead of <batchChangeSet>. The <method> tag is missing as the operation is always GET. you can also add headers as seen in case for Entity 'Product'. The response to the above request is as seen below.
The current OData V2 Adapter in SAP Cloud Platform Integration supports POST, PUT, MERGE,GET and DELETE HTTP operations in batch. Using the POST operation, you can insert ‘n’ number of new entities in your OData backend, using PUT/MERGE operation you can update entities and DELETE operation to delete entities. We will visit POST and PUT examples in the following sections.
To know more on how the OData Service supports batch you can go through OData documentation, http://www.odata.org/documentation/odata-version-2-0/batch-processing/ which will give you a brief overview on structure and format of a batch request.
Request Body for Batch Operation
The request body of batch contains an ordered sequence of operations to be performed. These modifying operations are called ‘ChangeSet’. ChangeSets are atomic units that contain the entity on which create, update or delete operations are to be performed. The ChangeSets can also contain an unordered sequence of operations. All changes in a ChangeSet must be all processed successfully or none of them. Implementation of ChangeSet varies with different services.
Here’s an example of the request body in SAP Cloud Platform Integration for ‘Batch Request’ which is denoted by a <batchParts> tag. This contains 2 ‘ChangeSets’ denoted by <batchChangeSet> tags. It is mandatory to have at least one ChangeSet in a batch request payload.
The <method> tag denotes which operation is to be performed. Iin this case, the operation is POST and the entity set is Categories. These tags are mandatory to be sent in all batch operations. There are a few optional tags like <headers> to send custom headers to ChangeSets. The <uri> tag is used to mention the exact entity on which you wish to perform the operation. It is mandatory in case of DELETE operation. Remember to add the key and its value when using <uri> tag for DELETE, MERGE or UPDATE operations.
Configuring Batch in the OData Adapter
We are using the OData Demo V2 service, http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc in this example.
- Select OData as the Receiver Adapter; screenshot of the adapter specific details is as seen below.
- Modelling Resource Path; connect to the service by adding a new system
- Enable ‘Batch Processing’ by switching to ON, after selecting the entity and operation of your choice. Here, entity is ‘Categories’ and Operation is ‘POST’.
- Click OK. Here’s a screenshot of how the adapter settings should look after modelling. Notice that the ‘Enable Batch Processing’ checkbox is enabled by default.
Note: Soon after modelling the operation, XSD file is generated and a message is displayed based on the fields selected by you in Step 3. This is to help you to map the request payload when using message mapping.
Understanding the Batch Response
Here’s an example of how the batch response looks like.
The first ChangeSet is the generic format and the second ChangeSet is an actual response received for the above request. Each individual ChangeSet response can be viewed under <batchChangeSetResponse> tag, which is followed by the <headers> tag. This <headers> tag denotes response headers from the OData Endpoint.
The <statusCode> and <statusInfo> gives us the HTTP status code and status line details respectively. <body> tag shows us the successfully inserted response in case of insert operations. It is empty for update operations. An error response is shown if either of the operations is unsuccessful. However, in the SAP Cloud Platform Integration’s message processing log (MPL), you will see that the status is COMPLETED. This is because, the batch requested has been accepted successfully by the OData server and it has provided a response.
Integration Flow with OData Batch
Scenario 1:With timer triggered event- Creating two new categories with body specified in the content modifier
The timer is configured to run once; the content modifier contains the request body. The response is captured in SFTP Server.
An example of batch request body is as seen below; we are inserting 2 new Categories with ID ‘25’ and ‘26’.
<batchParts>
<batchChangeSet>
<batchChangeSetPart>
<method>POST</method>
<Categories>
<Category>
<Name>Example1</Name>
<ID>25</ID>
</Category>
</Categories>
</batchChangeSetPart>
</batchChangeSet>
<batchChangeSet>
<batchChangeSetPart>
<method>POST</method>
<Categories>
<Category>
<Name>Example2</Name>
<ID>26</ID>
</Category>
</Categories>
</batchChangeSetPart>
</batchChangeSet>
</batchParts>
The output as observed in SFTP server is
<batchPartResponse>
<batchChangeSetResponse>
<batchChangeSetPartResponse>
<headers>
<Accept-Language></Accept-Language>
<DataServiceVersion>1.0;</DataServiceVersion>
<Location>http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc/Categories(25)</Location>
<Accept></Accept>
<Content-Type>application/atom+xml;charset=utf-8</Content-Type>
<Cache-Control>no-cache</Cache-Control>
</headers>
<statusCode>201</statusCode>
<body>
<Name>Example1</Name>
<ID>25</ID>
</body>
<contentId/>
<statusInfo>Created</statusInfo>
</batchChangeSetPartResponse>
</batchChangeSetResponse>
<batchChangeSetResponse>
<batchChangeSetPartResponse>
<headers>
<Accept-Language></Accept-Language>
<DataServiceVersion>1.0;</DataServiceVersion>
<Location>http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc/Categories(26)</Location>
<Accept></Accept>
<Content-Type>application/atom+xml;charset=utf-8</Content-Type>
<Cache-Control>no-cache</Cache-Control>
</headers>
<statusCode>201</statusCode>
<body>
<Name>Example2</Name>
<ID>26</ID>
</body>
<contentId/>
<statusInfo>Created</statusInfo>
</batchChangeSetPartResponse>
</batchChangeSetResponse>
</batchPartResponse>
Scenario 2: Using XSD generated to form the batch request payload
SOAP adapter triggers the call to the OData Endpoint. The response is sent to SFTP server via SFTP adapter. The target mapping used is the XSD generated from the OData Receiver.
Batch request body is shown below. The first entity should be successfully updated and the second one should fail because there is no Category(123) currently in the OData server.
<batchParts>
<batchChangeSet>
<batchChangeSetPart>
<method>PUT</method>
<Categories>
<Category>
<Name>NewExample1</Name>
<ID>25</ID>
</Category>
</Categories>
</batchChangeSetPart>
</batchChangeSet>
<batchChangeSet>
<batchChangeSetPart>
<method>PUT</method>
<Categories>
<Category>
<Name> NewExample2</Name>
<ID>123</ID>
</Category>
</Categories>
</batchChangeSetPart>
</batchChangeSet>
</batchParts>The output as observed in SFTP server is
<batchPartResponse>
<batchChangeSetResponse>
<batchChangeSetPartResponse>
<headers>
<Accept-Language/>
<DataServiceVersion>1.0;</DataServiceVersion>
<Accept/>
<Cache-Control>no-cache</Cache-Control>
</headers>
<statusCode>204</statusCode>
<body/>
<contentId/>
<statusInfo>No Content</statusInfo>
</batchChangeSetPartResponse>
</batchChangeSetResponse>
<batchChangeSetResponse>
<batchChangeSetPartResponse>
<headers>
<Accept-Language/>
<DataServiceVersion>1.0;</DataServiceVersion>
<Accept/>
<Content-Type>application/xml</Content-Type>
</headers>
<statusCode>400</statusCode>
<body><?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
<code></code>
<message xml:lang="en-US">Error processing request stream. The URI specified is not valid.</message>
<innererror>
<message>Invalid Uri specified. The query 'DataServiceProvider.DSPLinqQuery`1[DataServiceProvider.DSPResource]' must refer to a single resource</message>
<type>System.ArgumentException</type>
<stacktrace> at DataServiceProvider.DSPUpdateProvider.GetResource(IQueryable query, String fullTypeName) in c:\TFS\OData\Main\ndp\fx\src\DataWeb\ODataSDK\services.odata.org\DataServiceProvider\DSPUpdateProvider.cs:line 276&#xD;
at System.Data.Services.UpdatableWrapper.GetResource(IQueryable query, String fullTypeName)</stacktrace>
</innererror>
</error></body>
<contentId/>
<statusInfo>Bad Request</statusInfo>
</batchChangeSetPartResponse>
</batchChangeSetResponse>
</batchPartResponse>
Scenario 3: Batch Request with multiple entries in one ChangeSet
The request payload that we will be using for this scenario is as seen below.
<batchParts>
<batchChangeSet>
<batchChangeSetPart>
<method>POST</method>
<Categories>
<Category>
<Name>Create Example 1</Name>
<ID>16</ID>
</Category>
</Categories>
</batchChangeSetPart>
<batchChangeSetPart>
<method>POST</method>
<Categories>
<Category>
<Name>Create Example 2</Name>
<ID>17</ID>
</Category>
</Categories>
</batchChangeSetPart>
</batchChangeSet>
<batchChangeSet>
<batchChangeSetPart>
<method>PUT</method>
<Categories>
<Category>
<Name>Updated Example 1</Name>
<ID>16</ID>
</Category>
</Categories>
</batchChangeSetPart>
<batchChangeSetPart>
<method>PUT</method>
<Categories>
<Category>
<Name>Updated Example 2</Name>
<ID>17</ID>
</Category>
</Categories>
</batchChangeSetPart>
</batchChangeSet>
</batchParts>
Notice that, the request payload contains two ordered ChangeSets(batchChangeSet).
The first ChangeSet consists of two request to be performed(batchChangeSetPart), i.e creation of ID 16 and 17. If either of the 'batchChangeSetPart' fails neither will be completed. The rollback is performed by the OData Server if failure occurs.
These 'batchChangeSetPart' are unordered, meaning that any batchChangeSetPart can be performed first irrespective of the sequence in the payload.
The second ChangeSet consists of two update request of ID 16 and 17.
Successful response for the above request payload can be seen below. As of 2.27.0 version of Cloud Platform Integration, we allow only one response to be listed in the payload for each ChangeSet.
<batchPartResponse>
<batchChangeSetResponse>
<batchChangeSetPartResponse>
<headers>
<Accept-Language/>
<DataServiceVersion>1.0;</DataServiceVersion>
<Location>http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc/Categories(17)</Location>
<Accept/>
<Content-Type>application/atom+xml;charset=utf-8</Content-Type>
<Cache-Control>no-cache</Cache-Control>
</headers>
<statusCode>201</statusCode>
<body>
<Name>Create Example 2</Name>
<ID>17</ID>
</body>
<contentId/>
<statusInfo>Created</statusInfo>
</batchChangeSetPartResponse>
</batchChangeSetResponse>
<batchChangeSetResponse>
<batchChangeSetPartResponse>
<headers>
<Accept-Language/>
<DataServiceVersion>1.0;</DataServiceVersion>
<Accept/>
<Cache-Control>no-cache</Cache-Control>
</headers>
<statusCode>204</statusCode>
<body/>
<contentId/>
<statusInfo>No Content</statusInfo>
</batchChangeSetPartResponse>
</batchChangeSetResponse>
</batchPartResponse>
Scenario 4: Batch GET Request
Batch GET request requires a request payload like other Batch operations but the XML structure offers. Below is an example of a request.
<batchParts>
<batchQueryPart>
<method>GET</method>
<headers>
<header>
<headerName>dummy</headerName>
<headerValue>dummy</headerValue>
</header>
</headers>
<uri>./Products?$top=1</uri>
</batchQueryPart>
<batchQueryPart>
<method>GET</method>
<uri>Suppliers</uri>
</batchQueryPart>
<batchQueryPart>
<method>GET</method>
<uri>Categories(0)</uri>
</batchQueryPart>
<batchQueryPart>
<method>GET</method>
<uri>GetProductsByRating?rating=4</uri>
</batchQueryPart>
</batchParts>
In this example, we are performing 4 different GET operations, i.e. reading the first 'Product', getting all the 'Suppliers', reading 'Categories' with for ID 0 and performing function import 'GetProductsByRating' respectively. <uri> tag is necessary for Batch GET. You can also see the XML tag is <batchQueryPart> instead of <batchChangeSet>. The <method> tag is missing as the operation is always GET. you can also add headers as seen in case for Entity 'Product'. The response to the above request is as seen below.
<batchPartResponse>
<batchQueryPartResponse>
<headers>
<Cache-Control>no-cache</Cache-Control>
<Accept></Accept>
<Accept-Language></Accept-Language>
<DataServiceVersion>2.0;</DataServiceVersion>
<Content-Type>application/atom+xml;charset=utf-8</Content-Type>
</headers>
<statusInfo>OK</statusInfo>
<contentId/>
<body>
<Products>
<Product>
<Price>2.5</Price>
<Rating>4</Rating>
<DiscontinuedDate/>
<ID>0</ID>
<ReleaseDate>1992-01-01T00:00:00.000</ReleaseDate>
</Product>
</Products>
</body>
<statusCode>200</statusCode>
</batchQueryPartResponse>
<batchQueryPartResponse>
<headers>
<Cache-Control>no-cache</Cache-Control>
<Accept></Accept>
<Accept-Language></Accept-Language>
<DataServiceVersion>1.0;</DataServiceVersion>
<Content-Type>application/atom+xml;charset=utf-8</Content-Type>
</headers>
<statusInfo>OK</statusInfo>
<contentId/>
<body>
<Suppliers>
<Supplier>
<Address>
<State>WA</State>
<ZipCode>98074</ZipCode>
<Street>NE 228th</Street>
<Country>USA</Country>
<City>Sammamish</City>
</Address>
<Concurrency>0</Concurrency>
<ID>0</ID>
<Name>Exotic Liquids</Name>
</Supplier>
<Supplier>
<Address>
<State>WA</State>
<ZipCode>98052</ZipCode>
<Street>NE 40th</Street>
<Country>USA</Country>
<City>Redmond</City>
</Address>
<Concurrency>0</Concurrency>
<ID>1</ID>
<Name>Tokyo Traders</Name>
</Supplier>
</Suppliers>
</body>
<statusCode>200</statusCode>
</batchQueryPartResponse>
<batchQueryPartResponse>
<headers>
<Cache-Control>no-cache</Cache-Control>
<Accept></Accept>
<Accept-Language></Accept-Language>
<DataServiceVersion>1.0;</DataServiceVersion>
<Content-Type>application/atom+xml;charset=utf-8</Content-Type>
</headers>
<statusInfo>OK</statusInfo>
<contentId/>
<body>
<Categories>
<Category>
<ID>0</ID>
<Name>Food</Name>
</Category>
</Categories>
</body>
<statusCode>200</statusCode>
</batchQueryPartResponse>
<batchQueryPartResponse>
<headers>
<Cache-Control>no-cache</Cache-Control>
<Accept></Accept>
<Accept-Language></Accept-Language>
<DataServiceVersion>2.0;</DataServiceVersion>
<Content-Type>application/atom+xml;charset=utf-8</Content-Type>
</headers>
<statusInfo>OK</statusInfo>
<contentId/>
<body>
<Products>
<Product>
<Price>2.5</Price>
<Rating>4</Rating>
<DiscontinuedDate/>
<ID>0</ID>
<ReleaseDate>1992-01-01T00:00:00.000</ReleaseDate>
</Product>
</Products>
</body>
<statusCode>200</statusCode>
</batchQueryPartResponse>
</batchPartResponse>
- SAP Managed Tags:
46 Comments
