I’d like to give acknowledgement and thanks to my colleague Edmund Chua for contributing to this blog post. The content is based on our past integration experiences, and specifically our experience on a customer project which involves retail execution (Visit Planning, Activities, Surveys) using C4C and a 3rd party handheld solution in a high volume scenario (expected daily volume in the millions of records).

In this blog post, we’ll compare the 3 different web service integration methods (A2A, A2X, ODATA) for 2 common scenarios: Retrieving Data and Creating/Updating Data. We’ll also provide our recommendation on the type of web service to use in high volume scenarios and also share some lessons learned.

Integration Methodology Comparison

There are 3 integration methods that were analyzed for best fit:

A2X (Application to X users)

  • SOAP based web service, Synchronous execution

A2A (Application to Application)

  • SOAP based web service, Asynchronous execution

ODATA (Open Data Protocol)

  • REST based web service, Synchronous execution

Scenario 1: Retrieving Data

A2X Method

Using A2X to retrieve data requires submitting an XML formatted request payload to a service endpoint. The following is an A2X example of retrieving a specific survey by UUID:

Service Endpoint: https://myXXXXXX.crm.ondemand.com/sap/bc/srt/scs/sap/queryquestionnairevaluationcol

/wp-content/uploads/2016/07/a2x_retrieve_994766.png

The selection parameters are defined within the XML request body and the response is returned in XML format.

Using A2X Method for Retrieving Data:

Pros

Cons

  • Established services, stable.
  • Paging is available.
  • Associated entity (level 2), such as TextCollections and BTDReference, are provided in the result automatically.  The retrieval of these associated entities is efficient.
  • Limited to only query parameters available in the A2X web services.
  • No support for JSON format
  • Uses SOAP protocol (not as lightweight as ODATA)

ODATA Method

Using ODATA to retrieve data requires appending request parameters at the end of the service endpoint URL and submitting an HTTP GET request for the URL.

The following is an example of a request URL to retrieve Activity Worklist Items with various filters:

https://myXXXXXXcrm.ondemand.com/sap/c4c/odata/cust/v1/XX/ActivityWorklistItemCollection?$inlinecount=allpages&$filter=(ParentTypeCode eq ’12’) and (ParentGroupCode eq ‘0027’) and (ParentScheduledStartDate ge datetime’2016-04-27T00:00:00′) and (ActivityLifeCycleStatusCode eq ‘1’) and (ActivityTypeCode eq ‘542’) and (ActivityWorklistItemTypeCode eq ’01’) &$expand=ActivityAttachmentFolder,ActivityTextCollection

/wp-content/uploads/2016/07/odata_retrieve_994848.png

Unlike A2X, there is no request body that is sent to the service endpoint. All parameters and filters are embedded directly in the request URL itself.

The response can also be formatted as JSON using the directive $format=json, which returns a more compact response compared to an XML formatted response.

/wp-content/uploads/2016/07/odata_retrieve_response_994849.png

Using ODATA Method for Retrieving Data:

Pros

Cons

  • Established services, stable and lightweight.
  • Paging is available.
  • All fields in the ODATA object can be used as query parameters.  Greater flexibility in query design.
  • Supports XML and JSON
  • When the field to be filtered is in level 2 (such as dependency object/association), this field cannot be used.  Filters can only be applied to top level fields. [UpdateMustafa Saglam provided a workaround in the comments section]
  • For certain scenarios, such as filtering of codelists via Context, ODATA cannot be used due to Context not available.
  • Associated entities (level 2), such as TextCollection and BTDReferences, need to be added to the ODATA and then navigated using $expand.  Navigation incurs C4C performance overheads, and is not efficient for high volumes.


The choice of using ODATA or A2X for retrieving data depends on the business requirement.

For high volume data retrieval, ODATA is recommended for being lightweight compared to SOAP and also for its support of JSON formatted responses.

Scenario 2: Create/Update Data:

A2X Method

The following is an example of an A2X request to update 2 Task activities:

Service Endpoint: https://myXXXXXX.crm.ondemand.com/sap/bc/srt/scs/sap/managetaskactivityin

/wp-content/uploads/2016/07/a2x_create_994850.png

A2X allows you to submit updates in batches for better performance. In the simple example above, the system successfully processed both updates within the same request. In the scenario where one of the activities fails to update (i.e. object is already locked, invalid ID, etc.), then the entire request will fail. This is not ideal for high volume scenarios because just a single bad request in the batch can render the rest of the requests in the batch to also fail.

A2X Method for Creating/Updating Data:

Pros

Cons

  • Calling systems have full flexibility to handle all success and errors.
  • Official method for integration between C4C and Third Party systems.
  • Covers more business objects, so more comprehensive.
  • Calling system has to wait for reply, thus consuming more resources on the calling system.
  • Calling system can send high volumes anytime, and once C4C receive the input it has to process them immediately.  C4C cannot control the load, and overloading can occur.
  • Once the input contains one bad data, the whole call will fail.
  • Only technical errors are shown in Web Service Message monitor.  Limited monitoring available.


A2A Method

Similar to A2X requests, the A2A request for creating/updating entities requires an XML formatted payload to be sent to a C4C service endpoint. The following is an example request for creating a Task Activity using A2A:

Service Endpoint: https://myXXXXXX.crm.ondemand.com/sap/bc/srt/scs/sap/activityreplicationin

/wp-content/uploads/2016/07/a2a_create_994851.png

Since this is an A2A web service call (asynchronous), there will not be a response payload from C4C after your request is received. C4C will return an HTTP 202 Accepted header that signals that the request was received however the response will be empty. To view any success or error messages, you will need to view the log entries in C4C’s Web Service Message Monitoring Tool (Administrator -> Web Service Message Monitoring).

If the request was processed successfully, you will find a success log entry for the request:

/wp-content/uploads/2016/07/1_994852.png

If an error occurred, an error entry will be logged in the message monitor:

/wp-content/uploads/2016/07/2_994856.png

To view additional details of the error, you can open the Error Log for the request:

/wp-content/uploads/2016/07/3_994857.png

Unlike A2X requests where the calling system is required to wait for C4C to execute the actions in the request, A2A requests are asynchronous

A2A Method for Creating/Updating Data:

Pros

Cons

  • Asynchronous.  Calling system doesn’t have to wait for a response from C4C, thus less resource consumption on calling system.
  • Requests are queued in C4C for processing, producing better load handling in C4C.
  • Monitoring of success and failure available from C4C’s Web Service Message Monitoring.
  • When a single A2A call contains good and bad data, C4C can split the input to process the good data, and fail the bad data.
  • These interfaces are for integration between C4C and SAP ERP/CRM, so they are not publicly available for integration between C4C and Third Party systems.
  • No documentation available, and everything is a black box, so use at your own risk. SAP will not support you if you have errors using it for Third Party integration.


ODATA Method

The following is an example of creating a new contact person using ODATA:

Service: https://myXXXXXX.crm.ondemand.com/sap/c4c/odata/v1/c4codata/ContactCollection

/wp-content/uploads/2016/07/4_994858.png

Using ODATA Method for Creating/Updating Data:

Pros

Cons

  • Light weight and executed using HTTPS protocol.
  • If the input contains bad data, these can be skipped to process only the good ones, provided that the data are in different changesets.
  • All fields in the ODATA object are available for create/update, thus greater flexibility.
  • No monitoring available in C4C.
  • Response payload doesn’t provide descriptive messages for either successful or failed requests
  • Not stable, so not recommended until after 1608 (tentative).

The choice of using A2A, A2X, or ODATA depends on the business requirement.  For high volume create/update requests, A2A is recommended due to it being an asynchronous service.  We use A2X only when A2A is not available.

However, a caveat on using A2A is that it’s not officially available for Third Party integration (i.e. integrating with a non-SAP system). If you encounter issues with an A2A web service after go-live, there is no commitment from SAP to provide support for your A2A solution, so be aware!

Lessons Learned

Design Considerations for High Volume

To improve efficiency in high volume scenarios, the below principles were used in our interface design:

  1. Reduce the number of ODATA calls.  This can be done by using ToParent property in the ODATA, and merging this ToParent association.  For example, by merging ToParent in WorklistItem, we can call WorklistItem directly to get the attributes in both Worklist and WorklistItem, instead of calling Worklist then WorklistItem.
  2. Avoid the use of navigations in ODATA as these reduce performance in high volume cases. Have all the data available in level 1 of the ODATA object.  This is achieved by merging associations.
  3. Instead of using association, another alternative is to data join.  However, do the join outside C4C, i.e. extract the data and join in your external database.
  4. Minimize extraction of duplicate data.

ODATA: Making Batch Calls

ODATA allows performing several operations within a single request using the $batch directive.

You have the option of encapsulating each operation in its own change-set or lump all operations into a single change-set. The issue with the latter is that, unlike A2A, C4C will not split the batch request if a single operation of the batch call fails. C4C will return a single failed response and it will be up to your application to fix/remove the failing request and to resubmit the batch request again for reprocessing. If using multiple change-sets, C4C will return a response for each change-set in the batch call.

ODATA: Retrieve Data in Batches

When retrieving large volumes of data, we like to retrieve in batches for better management of resources.

One way is to specify the size of each retrieval, such as getting 1000 records each time.   This can be done using the $top and $skip.

To get the first 100 records:

https://…./ActivityCollection?$top=100

Then the next 100 records:

https://…./ActivityCollection?$top=100&$skip100

When it is not possible to use $top and $skip (for example in BODS, all data is retrieved before $top is applied, thus making $top ineffective), another way is to use range, such as retrieve (1<=ID<=1000), then (1001<=ID<=2000).

https://…./ActivityCollection?$filter=(ID ge ‘100’ and ID le ‘300’)

For range, due to a limitation in the ODATA framework, gt and lt on the same field does not work. Therefore, we should always only use ge and le.

The last way is to retrieve data using different key values, such as when Type = 1, then when Type = 2, and so on.

https://……../ActivityCollection?$filter=(Type ge ‘1’)

A2A Web Services: WS-Addressing

For asynchronous web service calls (A2A), we need to use WS-Addressing to execute the calls.

Where middleware cannot support WS-Addressing, we need to modify the XML input headers sent.

From the WSDL, look for wsdl:binding where the type is the name of the A2A call.  Get the soapAction value.

/wp-content/uploads/2016/07/5_994865.png

In the header of the web service call, define the soapenv:Header manually as below.  The soapAction will be the wsa:Action.  Give a unique MessageID in UUID format.  Note that this MessageID must be unique for every call.

/wp-content/uploads/2016/07/6_994866.png

Use the same MessageID value for the MessageHeader.ID sections in the body of the web service call.

A2A Web Services:  BODS issues with SOAP version 1.2 binding

When using BODS (SAP Business Objects Data Services), the XML parser is not able to identify the soapAction element if the WSDL is based on SOAP version 1.2. C4C’s WSDLs contain both SOAP version 1.1 and 1.2 bindings. To avoid issues, ensure that your the WSDL contains the SOAP version 1.1 binding.

/wp-content/uploads/2016/07/ddd_999551.png

A2A Web Services: Testing with SOAPUI

If you are using SOAPUI to test A2A web services, you will need to make sure the WS-A addressing is enabled and the MessageID is generated automatically:

/wp-content/uploads/2016/07/8_994871.png

BODS Performance – Truncate WSDL

SAP Business Object Data Services loads the entire WDSL content in memory for each request that is made. For the elements that are not used, BODS will set the element value to ‘NULL’ which impacts performance in terms of the memory resources required.

To reduce the memory footprint and improve performance, it is recommended to comment out the unnecessary elements in the WSDL.

Resources

SAP Cloud for Customer OData API Developer’s Guide

https://github.com/SAP/C4CODATAAPIDEVGUIDE

SAP ODATA API Doc

http://help.sap.com/saphelp_sapcloudforcustomer/en/ODATA_APIs/index.html#1204fcf690e346ff859b49a7e02b49c0.html

ODATA V2.0 Documentation (SAP uses V2.0)

http://www.odata.org/documentation/odata-version-2-0/

C4C ODATA Examples

https://github.com/venkyvb/c4c-api

Postman – ODATA Testing Tool

http://www.getpostman.com

SOAPUI – SOAP Testing Tool

https://www.soapui.org

To report this post you need to login first.

6 Comments

You must be Logged on to comment or reply to a post.

  1. Venki Balakrishnan

    Great write up. One comment regarding the “A2A Web Services:  soapAction” section. C4C WSDL’s contain both SOAP 1.1 and SOAP 1.2 bindings. You seem to be using the SOAP 1.2 bindings which is basically the “binding_SOAP12”. The SOAP 1.1 bindings would be in the format that you have described above after the correction (binding name would be simply “binding”). From what you describe BODS works with the SOAP 1.1 bindings.

    (1) 
    1. Michael Mao Post author

      Hi Venki, thanks for the clarification. I’ve updated that section to make it clear about the issue with using BODS and the SOAP version 1.2 binding

      (0) 
  2. Mustafa Saglam

    Hi Michael, Edmund, thank you both for contributing such an excellent resource to our community. I particularly liked your pros & cons for each option. Obviously, a lot of work and thinking went into it.

    Couple of comments regarding your notes on OData:

    • Under “Using ODATA Method for Retrieving Data:” you correctly stated that C4C OData API currently doesn’t support $filter on sub-entity properties when the main entity is being queried. However, many times it is possible to work around this limitation by using the dual parent-child relationship that exist between most of our entities. Basically, to implement this workaround, one would query the sub-entity collection, filter on the sub-entity property and use $expand to access the main-entity properties.
    • Under “ODATA: Making Batch Calls” you are specifically referring to a using a single change-set. This approach does have some weaknesses as it relates to handling errors in individual create or update requests. However, I wanted to mention here that you do have a choice and can send each of your create/update request on its own change-set. In this case, each unique change-set will have its own response.
    • Furthermore, while I cannot share too much detail here nor can I share a timeline, I wanted to inform you that in the future we will also have some async capability available for our OData API which will make it even more preferable for data integration scenarios involving higher volumes.

    Best,

    Mustafa.

    (1) 
    1. Michael Mao Post author

      Thanks for your comments, Mustafa. I’ve updated the blog to include your comments. Much appreciation for the workaround, and looking forward to the new async capabilities in the ODATA API!

      (0) 

Leave a Reply