SAP Customer Data Cloud Integration via SAP CPI(OAuth based)

Introduction –

In this blog I would like to share how we can implement integration between SAP Customer data cloud(CDC) to SAP Marketing & Service Cloud using SAP Cloud platform integration(CPI) as Middleware.

Authentication we wanted to be OAuth based rather basic authentication as SAP CDC Identity sync comes out of the box.

We are using IdentitySync to push data out from SAP CDC to SAP CPI and we will focus on dataflow configuration in this blog.

Second part of this series will cover SAP CPI Iflow designs as well.

Design Approach –

SAP CDC & Marketing cloud has out of the box point to point integration. Where it directly calls SAP Marketing Cloud Odata services in IdentitySync dataflow.

You can refer to details of this integration in SAP CDC developers Wiki.

https://developers.gigya.com/display/GD/SAP+Marketing+Cloud

But there is no out of the box integration available between SAP CDC & Service Cloud and no standard way to have OAuth based authentication as well.

Solution –

We used SAP CPI as middleware and built our custom Iflows to direct the profiles data to SAP Service & Marketing endpoints.

We can achieve this through “External Generic” writer steps in the IdentitySync data flow.

To support OAuth based authentication we used “Record.evaluate” step in Dataflow and formatted it to pass it on to final writer steps.

Let’s start building the Dataflow for our integration.

Prerequisites –

• Whitelisting of SAP CPI end point urls – This is a mandatory step as we will be using “External generic” writer step in our integration. All the endpoints urls which we are going to use, should be whitelisted for your partner key(tenant).
• You need to raise a support ticket to SAP CDC product team to get end point urls whitelisted. Else you will get a standard error as below.

Endpoint url whitelisting error

Dataflow Configuration –

Step#1 –

Create your data flow by selecting the SAP Marketing outbound dataflow template. As we are going to use mostly similar steps “Rename” & ”Field.evaluate”, “Remove” to get the fields synced with other systems.

We will have standard dataflow populated as below.

Step #2– Now edit the “Reader steps” and adjust as per the need.

Step #2a–

• Write a select query to pick the fields you want to send to SAP Marketing or Service Cloud.
• Select Email Account in field “from”
• Enter DeltaField = LastUpdatedTimestamp

Note – Email accounts are selected when we want to send Consents & Preferences along with Profile information. By default it has “Accounts” selected and wont send Consents with profile information.

You can modify as per your field requirements Rename, Fields.Evaluate, Remove steps in the dataflow.

Step #3 – Remove last Writer step of SAP Marketing cloud and replace with an “External generic” steps and modify it as shown below to receive OAuth token from CPI end point.

Step #3a –

• Edit the step and enter SAP CPI end point url to get OAuth token as below.
• Enter Method as “POST”.
• Don’t forget to select checkbox “AddResponse”. By this you can add token response to next step.
• Maintain your header Authorization – header name & value(taken from SAP CPI)

Step #4 – Now we need to send this OAuth token received in last step to the next writer step. But we have an issue here as OAuth Token will not be constant or fixed length and will be updated in every response from SAP CPI.

To handle this, we need to first format the token and then pass it to final writer steps of SAP Marketing & Service.

We will write our own script to format the token and for that SAP CDC provides “Record.evaluate” step. In this step we can write our own custom script.

Step #4a – Edit your “Record.evalate” step and write the below script to format the token.

Note – You can also test your script by pressing button Test script and validate the output.

Step #5 – Now in the final step you need to call your SAP CPI endpoints for SAP Marketing & Service Iflows.

You can call both in the same iflow as below and pass the bearer token formatted in the last step.

Add 2 more “external.generic” steps and point them to SAP Marketing & Service cloud iflow urls.

Step #5a –

• In the url enter your SAP CPI Marketing cloud iflow end point url.
• Maintain Method as Post for this call.
• Enter Authorization and pass the value of token received in the last step.

Note – Don’t forget to maintain “maxConnections” to handle parallel processing of the records. It will help you execute your records without running into any memory issues. Like below common error.

Trace capacity issue

Step #6 – Final step is to schedule a job as per your defined regular interval to push profiles data from SAP CDC.

Step #7 – Define the parameter to run in defined frequency and maintain email address to get email on success and failure of the job.

You can find more information on 2^nd part of my next blog related to Iflow design we have built for SAP Marketing & Service cloud in SAP CPI.

Monitoring Job status & Error logs – 

You can select the dataflow from the list and from actions you can navigate to “status” to monitor the status of the job scheduled and check error logs.

Click on information to check the details of it.
Trace – It will display all the steps executed sequentially in your dataflow. Log level, step name and message output from it.

Step Metrics – It will display time taken by each step. Number of records input and output and error out record on each step.

Error section – This displays the detailed error and steps on which error occurred and count as well.

References:

• https://developers.gigya.com/display/GD/IdentitySync
• https://developers.gigya.com/display/GD/SAP+Marketing+Cloud
• https://developers.gigya.com/display/GD/IdentitySync+Custom+Scripts

Conclusion:

Hope this blog helps In integration of SAP Customer data cloud with SAP Marketing & Service Cloud via SAP Cloud Platform Integration Suite.

Thanks to my colleagues – Carlos Raimundo , Garvit Agrawal, Manu Gupta

Regards,

Gaurav Singh Jat