You can perform batch operations in the OData V2 Adapter for SAP Cloud Platform Integration. Batch, as its name suggests, implies that multiple requests are clubbed into one HTTP call. What this means is that if you are updating 100 records, instead of making 100 PUT or update request, the adapter would just make one call to the server with all the essential data. This would greatly improve time taken to achieve such requirements, also improving the performance.

The current OData V2 Adapter in SAP Cloud Platform Integration supports POST, PUT, MERGE and DELETE HTTP operations in batch. Using the POST operation, you can insert ‘n’ number of new entities in your OData backend, using PUT/MERGE operation you can update entities and DELETE operation to delete entities. We will visit POST and PUT examples in the following sections.

The OData V2 Adapter as of version 2.27.0 does not support QUERY or READ HTTP operation in batch.

To know more on how the OData Service supports batch you can go through OData documentation, http://www.odata.org/documentation/odata-version-2-0/batch-processing/ which will give you a brief overview on structure and format of a batch request.

 

Request Body for Batch Operation

The request body of batch contains an ordered sequence of operations to be performed. These modifying operations are called ‘ChangeSet’. ChangeSets are atomic units that contain the entity on which create, update or delete operations are to be performed. The ChangeSets can also contain an unordered sequence of operations. All changes in a ChangeSet must be all processed successfully or none of them. Implementation of ChangeSet varies with different services.

Here’s an example of the request body in SAP Cloud Platform Integration for ‘Batch Request’ which is denoted by a <batchParts> tag. This contains 2 ‘ChangeSets’ denoted by <batchChangeSet> tags. It is mandatory to have at least one ChangeSet in a batch request payload.

The <method> tag denotes which operation is to be performed. Iin this case, the operation is POST and the entity set is Categories. These tags are mandatory to be sent in all batch operations. There are a few optional tags like <headers> to send custom headers to ChangeSets. The <uri> tag is used to mention the exact entity on which you wish to perform the operation. It is mandatory in case of DELETE operation.

Configuring Batch in the OData Adapter

We are using the OData Demo V2 service, http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc in this example.

  1. Select OData as the Receiver Adapter; screenshot of the adapter specific details is as seen below.
  2. Modelling Resource Path; connect to the service by adding a new system
  3. Enable ‘Batch Processing’ by switching to ON, after selecting the entity and operation of your choice. Here, entity is ‘Categories’ and Operation is ‘POST’.
  4. Click OK. Here’s a screenshot of how the adapter settings should look after modelling. Notice that the ‘Enable Batch Processing’ checkbox is enabled by default.

 

Note: Soon after modelling the operation, XSD file is generated and a message is displayed based on the fields selected by you in Step 3. This is to help you to map the request payload when using message mapping.

 

Understanding the Batch Response

Here’s an example of how the batch response looks like.

The first ChangeSet is the generic format and the second ChangeSet is an actual response received for the above request. Each individual ChangeSet response can be viewed under <batchChangeSetResponse> tag, which is followed by the <headers> tag. This <headers> tag denotes response headers from the OData Endpoint.

The <statusCode> and <statusInfo> gives us the HTTP status code and status line details respectively. <body> tag shows us the successfully inserted response in case of insert operations. It is empty for update operations. An error response is shown if either of the operations is unsuccessful. However, in the SAP Cloud Platform Integration’s message processing log (MPL), you will see that the status is COMPLETED. This is because, the batch requested has been accepted successfully by the OData server and it has provided a response.

 

Integration Flow with OData Batch

Scenario 1:With timer triggered event- Creating two new categories with body specified in the content modifier

The timer is configured to run once; the content modifier contains the request body. The response is captured in SFTP Server.

An example of batch request body is as seen below; we are inserting 2 new Categories with ID ‘25’ and ‘26’.

<batchParts>
	<batchChangeSet>
		<batchChangeSetPart>
			<method>POST</method>
			<Categories>
				<Category>
					<Name>Example1</Name>
					<ID>25</ID>
				</Category>
			</Categories>
		</batchChangeSetPart> 
	</batchChangeSet>
	<batchChangeSet>
		<batchChangeSetPart>
			<method>POST</method>
			<Categories>
				<Category>
					<Name>Example2</Name>
					<ID>26</ID>
				</Category>
			</Categories>
		</batchChangeSetPart> 
	</batchChangeSet>
</batchParts>

The output as observed in SFTP server is

<batchPartResponse>
    <batchChangeSetResponse>
      <batchChangeSetPartResponse>
        <headers>
          <Accept-Language></Accept-Language>
          <DataServiceVersion>1.0;</DataServiceVersion>
          <Location>http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc/Categories(25)</Location>
          <Accept></Accept>
          <Content-Type>application/atom+xml;charset=utf-8</Content-Type>
          <Cache-Control>no-cache</Cache-Control>
        </headers>
        <statusCode>201</statusCode>
        <body>
          <Name>Example1</Name>
          <ID>25</ID>
        </body>
        <contentId/>
        <statusInfo>Created</statusInfo>
      </batchChangeSetPartResponse>
    </batchChangeSetResponse>
    <batchChangeSetResponse>
      <batchChangeSetPartResponse>
        <headers>
          <Accept-Language></Accept-Language>
          <DataServiceVersion>1.0;</DataServiceVersion>
          <Location>http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc/Categories(26)</Location>
          <Accept></Accept>
          <Content-Type>application/atom+xml;charset=utf-8</Content-Type>
          <Cache-Control>no-cache</Cache-Control>
        </headers>
        <statusCode>201</statusCode>
        <body>
          <Name>Example2</Name>
          <ID>26</ID>
        </body>
        <contentId/>
        <statusInfo>Created</statusInfo>
      </batchChangeSetPartResponse>
    </batchChangeSetResponse>
  </batchPartResponse>

 

Scenario 2: Using XSD generated to form the batch request payload

SOAP adapter triggers the call to the OData Endpoint. The response is sent to SFTP server via SFTP adapter. The target mapping used is the XSD generated from the OData Receiver.

Batch request body is shown below. The first entity should be successfully updated and the second one should fail because there is no Category(123) currently in the OData server.

<batchParts>
	<batchChangeSet>
		<batchChangeSetPart>
			<method>PUT</method>
			<Categories>
				<Category>
					<Name>NewExample1</Name>
					<ID>25</ID>
				</Category>
			</Categories>
		</batchChangeSetPart> 
	</batchChangeSet>
	<batchChangeSet>
		<batchChangeSetPart>
			<method>PUT</method>
			<Categories>
				<Category>
					<Name> NewExample2</Name>
					<ID>123</ID>
				</Category>
			</Categories>
		</batchChangeSetPart> 
	</batchChangeSet>
</batchParts>

The output as observed in SFTP server is

<batchPartResponse>
	<batchChangeSetResponse>
		<batchChangeSetPartResponse>
			<headers>
				<Accept-Language/>
				<DataServiceVersion>1.0;</DataServiceVersion>
				<Accept/>
				<Cache-Control>no-cache</Cache-Control>
			</headers>
			<statusCode>204</statusCode>
			<body/>
			<contentId/>
			<statusInfo>No Content</statusInfo>
		</batchChangeSetPartResponse>
	</batchChangeSetResponse>
	<batchChangeSetResponse>
		<batchChangeSetPartResponse>
			<headers>
				<Accept-Language/>
				<DataServiceVersion>1.0;</DataServiceVersion>
				<Accept/>
				<Content-Type>application/xml</Content-Type>
			</headers>
			<statusCode>400</statusCode>
			<body>&lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot; standalone=&quot;yes&quot;?&gt;
&lt;error xmlns=&quot;http://schemas.microsoft.com/ado/2007/08/dataservices/metadata&quot;&gt;
  &lt;code&gt;&lt;/code&gt;
  &lt;message xml:lang=&quot;en-US&quot;&gt;Error processing request stream. The URI specified is not valid.&lt;/message&gt;
  &lt;innererror&gt;
    &lt;message&gt;Invalid Uri specified. The query &apos;DataServiceProvider.DSPLinqQuery`1[DataServiceProvider.DSPResource]&apos; must refer to a single resource&lt;/message&gt;
    &lt;type&gt;System.ArgumentException&lt;/type&gt;
    &lt;stacktrace&gt;   at DataServiceProvider.DSPUpdateProvider.GetResource(IQueryable query, String fullTypeName) in c:\TFS\OData\Main\ndp\fx\src\DataWeb\ODataSDK\services.odata.org\DataServiceProvider\DSPUpdateProvider.cs:line 276&amp;#xD;
   at System.Data.Services.UpdatableWrapper.GetResource(IQueryable query, String fullTypeName)&lt;/stacktrace&gt;
  &lt;/innererror&gt;
&lt;/error&gt;</body>
			<contentId/>
			<statusInfo>Bad Request</statusInfo>
		</batchChangeSetPartResponse>
	</batchChangeSetResponse>
</batchPartResponse>

 

Scenario 3: Batch Request with multiple entries in one ChangeSet

The request payload that we will be using for this scenario is as seen below.

<batchParts>
	<batchChangeSet>
		<batchChangeSetPart>
			<method>POST</method>
			<Categories>
				<Category>
					<Name>Create Example 1</Name>
					<ID>16</ID>
				</Category>
			</Categories>
		</batchChangeSetPart> 
		<batchChangeSetPart>
			<method>POST</method>
			<Categories>
				<Category>
					<Name>Create Example 2</Name>
					<ID>17</ID>
				</Category>
			</Categories>
		</batchChangeSetPart>
	</batchChangeSet>
	<batchChangeSet>
		<batchChangeSetPart>
			<method>PUT</method>
			<Categories>
				<Category>
					<Name>Updated Example 1</Name>
					<ID>16</ID>
				</Category>
			</Categories>
		</batchChangeSetPart> 
		<batchChangeSetPart>
			<method>PUT</method>
			<Categories>
				<Category>
					<Name>Updated Example 2</Name>
					<ID>17</ID>
				</Category>
			</Categories>
		</batchChangeSetPart> 
	</batchChangeSet>
</batchParts>

Notice that, the request payload contains two ordered ChangeSets(batchChangeSet).

The first ChangeSet consists of two request to be performed(batchChangeSetPart), i.e creation of ID 16 and 17. If either of the ‘batchChangeSetPart’ fails neither will be completed. The rollback is performed by the OData Server if failure occurs.

These ‘batchChangeSetPart’ are unordered, meaning that any batchChangeSetPart can be performed first irrespective of the sequence in the payload.

The second ChangeSet consists of two update request of ID 16 and 17.

Successful response for the above request payload can be seen below. As of 2.27.0 version of Cloud Platform Integration, we allow only one response to be listed in the payload for each ChangeSet.

<batchPartResponse>
	<batchChangeSetResponse>
		<batchChangeSetPartResponse>
			<headers>
				<Accept-Language/>
				<DataServiceVersion>1.0;</DataServiceVersion>
				<Location>http://services.odata.org/(S(btmy2xygehkm1gfrnpezr4xx))/V2/OData/OData.svc/Categories(17)</Location>
				<Accept/>
				<Content-Type>application/atom+xml;charset=utf-8</Content-Type>
				<Cache-Control>no-cache</Cache-Control>
			</headers>
			<statusCode>201</statusCode>
			<body>
				<Name>Create Example 2</Name>
				<ID>17</ID>
			</body>
			<contentId/>
			<statusInfo>Created</statusInfo>
		</batchChangeSetPartResponse>
	</batchChangeSetResponse>
	<batchChangeSetResponse>
		<batchChangeSetPartResponse>
			<headers>
				<Accept-Language/>
				<DataServiceVersion>1.0;</DataServiceVersion>
				<Accept/>
				<Cache-Control>no-cache</Cache-Control>
			</headers>
			<statusCode>204</statusCode>
			<body/>
			<contentId/>
			<statusInfo>No Content</statusInfo>
		</batchChangeSetPartResponse>
	</batchChangeSetResponse>
</batchPartResponse>
To report this post you need to login first.

4 Comments

You must be Logged on to comment or reply to a post.

  1. Mark Bernabe

    Hi Sana,

    Thanks for the well explained blog! One thing we’ve noticed with the Batch Request is its performance. It is really slow compared to a normal OData request. For example, when creating Interactions and Interaction Contacts from external system, we use CUAN_IMPORT_SRV OData service. In the File based load – Interactions iFlow inside the SAP Hybris Marketing Cloud – file based data load package, Batch Request is used. However, it seems that we can also achieve the same results by just using the normal ImportHeader instead of Batch processing. In addition, it is a lot faster.

    So when is Batch Request recommended over the normal ImportHeaders and vice versa specifically for CUAN_IMPORT_SRV?

    (0) 
    1. Sana Faraz Post author

      Hi Mark,

      Thank you for your comment. From what I know, ImportHeader was added 3 years ago when batch was not available, it works by internally using deep insert. I also checked with Hybris colleagues and added a new example to this blog [Scenario 3: Batch Request with multiple entries in one ChangeSet] which would considerably improve the performance of batch calls.

       

      I am not an expert in SAP Hybris Marketing services but some differences that I could notice with service ‘CUAN_IMPORT_SRV’ are

      Batch ImportHeader
      Batch- POST call Deep Insert Operation- POST Call
      Can be used with all entities defined in the metadata Can only be used with entities ‘Contacts’ and ‘Interactions’ (navigations as defined in the metadata)
      Create,Merge,Update,Delete operations supported Only Create supported

      i would not be able to comment on the performance with ImportHeader but if you have huge chuck of data to be loaded, batch would be the suggested approach.

      For smaller data, as you mentioned Import Header might be better.

      (0) 
  2. Mark Bernabe

    Hi Sana,

    Thanks for taking the time to answer my question. It’s really helpful and greatly appreciated. With regards to the performance, I think it’s the other way around. For huge volume of data, ImportHeader is faster compared to using BatchRequest. From our tests, a batch of 30k Interactions and Contacts only took 4 mins to complete with ImportHeader as compared to using Batch that took more than 2 hours. Moreover, most of the time it fails because of timeout error.

    (0) 

Leave a Reply