Skip to Content
Technical Articles
Author's profile photo Gaurav Singh Jat

SAP Customer Data Cloud Integration via SAP CPI(OAuth based)

Introduction

In this blog I would like to share how we can implement integration between SAP Customer data cloud(CDC) to SAP Marketing & Service Cloud using SAP Cloud platform integration(CPI) as Middleware.

Authentication we wanted to be OAuth based rather basic authentication as SAP CDC Identity sync comes out of the box.

We are using IdentitySync to push data out from SAP CDC to SAP CPI and we will focus on dataflow configuration in this blog.

Second part of this series will cover SAP CPI Iflow designs as well.

Design Approach –

SAP CDC & Marketing cloud has out of the box point to point integration. Where it directly calls SAP Marketing Cloud Odata services in IdentitySync dataflow.

You can refer to details of this integration in SAP CDC developers Wiki.

https://developers.gigya.com/display/GD/SAP+Marketing+Cloud

But there is no out of the box integration available between SAP CDC & Service Cloud and no standard way to have OAuth based authentication as well.

Solution –

We used SAP CPI as middleware and built our custom Iflows to direct the profiles data to SAP Service & Marketing endpoints.

We can achieve this through “External Generic” writer steps in the IdentitySync data flow.

To support OAuth based authentication we used “Record.evaluate” step in Dataflow and formatted it to pass it on to final writer steps.

Let’s start building the Dataflow for our integration.

Prerequisites –

  • Whitelisting of SAP CPI end point urls – This is a mandatory step as we will be using “External generic” writer step in our integration. All the endpoints urls which we are going to use, should be whitelisted for your partner key(tenant).
  • You need to raise a support ticket to SAP CDC product team to get end point urls whitelisted. Else you will get a standard error as below.

Endpoint%20url%20whitelisting%20error

Endpoint url whitelisting error

Dataflow Configuration

 

Step#1 –

Create your data flow by selecting the SAP Marketing outbound dataflow template. As we are going to use mostly similar steps “Rename” & ”Field.evaluate”, “Remove” to get the fields synced with other systems.

Create%20Dataflow

We will have standard dataflow populated as below.

Step #2– Now edit the “Reader steps” and adjust as per the need.

Step #2a

  • Write a select query to pick the fields you want to send to SAP Marketing or Service Cloud.
  • Select Email Account in field “from”
  • Enter DeltaField = LastUpdatedTimestamp

Note – Email accounts are selected when we want to send Consents & Preferences along with Profile information. By default it has “Accounts” selected and wont send Consents with profile information.

You can modify as per your field requirements Rename, Fields.Evaluate, Remove steps in the dataflow.

Step #3 – Remove last Writer step of SAP Marketing cloud and replace with an “External generic” steps and modify it as shown below to receive OAuth token from CPI end point.

Step #3a –

  • Edit the step and enter SAP CPI end point url to get OAuth token as below.
  • Enter Method as “POST”.
  • Don’t forget to select checkbox “AddResponse”. By this you can add token response to next step.
  • Maintain your header Authorization – header name & value(taken from SAP CPI)

Step #4 – Now we need to send this OAuth token received in last step to the next writer step. But we have an issue here as OAuth Token will not be constant or fixed length and will be updated in every response from SAP CPI.

To handle this, we need to first format the token and then pass it to final writer steps of SAP Marketing & Service.

We will write our own script to format the token and for that SAP CDC provides “Record.evaluate” step. In this step we can write our own custom script.

Step #4a – Edit your “Record.evalate” step and write the below script to format the token.

Note – You can also test your script by pressing button Test script and validate the output.

Step #5 – Now in the final step you need to call your SAP CPI endpoints for SAP Marketing & Service Iflows.

You can call both in the same iflow as below and pass the bearer token formatted in the last step.

Add 2 more “external.generic” steps and point them to SAP Marketing & Service cloud iflow urls.

Step #5a

  • In the url enter your SAP CPI Marketing cloud iflow end point url.
  • Maintain Method as Post for this call.
  • Enter Authorization and pass the value of token received in the last step.

Note – Don’t forget to maintain “maxConnections” to handle parallel processing of the records. It will help you execute your records without running into any memory issues. Like below common error.

Trace%20capacity%20issue

Trace capacity issue

Step #6 – Final step is to schedule a job as per your defined regular interval to push profiles data from SAP CDC.

Step #7 – Define the parameter to run in defined frequency and maintain email address to get email on success and failure of the job.

You can find more information on 2nd part of my next blog related to Iflow design we have built for SAP Marketing & Service cloud in SAP CPI.

Monitoring Job status & Error logs – 

You can select the dataflow from the list and from actions you can navigate to “status” to monitor the status of the job scheduled and check error logs.

Click on information to check the details of it.
Trace – It will display all the steps executed sequentially in your dataflow. Log level, step name and message output from it.

Step Metrics – It will display time taken by each step. Number of records input and output and error out record on each step.

Error section – This displays the detailed error and steps on which error occurred and count as well.

References:

Conclusion:

Hope this blog helps In integration of SAP Customer data cloud with SAP Marketing & Service Cloud via SAP Cloud Platform Integration Suite.

Thanks to my colleagues – Carlos Raimundo , Garvit AgrawalManu Gupta

Regards,

Gaurav Singh Jat

Assigned Tags

      10 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Ashutosh Jain
      Ashutosh Jain

      Hi Gaurav

      Nicely articulated and too much useful as there is no standard way to connect C4C via CDC.

       

      Regards

      Ashutosh Jain

      Author's profile photo Bhavishya Mittal
      Bhavishya Mittal

      Excellent blog!!

      Author's profile photo Rupesh Banwari
      Rupesh Banwari

      Great Blog.. Thanks for sharing..

      Author's profile photo Sanjay Sharma
      Sanjay Sharma

      Very well explained, Great Blog!

      Author's profile photo Ritanshi Bansal
      Ritanshi Bansal

      Hi @gauravjat0405

      Just wanted to understand what is the reason you choose to use Oauth based authentication here. Because I am trying the same in my project as well, I used the external writer component and used the ilfow endpoint URL to trigger the iflow sending data from CDC to C4C. Therefore need and opinion is if it is the correct approach or should OAuth be used for such scenarios.

      Thanks

       

      Author's profile photo Gaurav Singh Jat
      Gaurav Singh Jat
      Blog Post Author

      Hi ritanshi bansal,

      Oauth or certificate based authentication are better secured connection than basic authentication.

      BR,

      Gaurav

      Author's profile photo Enrique Florian Gaviria
      Enrique Florian Gaviria

      Hi

       

      We already implement the dataflow follow  the obove  guide, nonetheless we got serious performance problems

      In marketing cloud we have a hard restriction related to API_MKT_CONTACT_SRV version 4  we must use  this version so in the middle we need use CPI to adapt  the CDC contact request before to send to marketing, this is why in  CDC in our Custom Data Flow we use a “datasource.write.external.generic”  to send the contacts. We notice that this component doesn´t have a batch parameter to group contacts before send them to CPI this could be avoid high volume calls to CPI and maybe increase the performance. At the same time we realize that the standard  stept “datasource.write.hybrismarketing”  already have this parameter

      Could you confirm us if its a recommendable practice group contacts before  send them in order to grant and acceptable performance and could you give us additional support to add this a batch parameter to our custom data flow  step “datasource.write.external.generic”

      Author's profile photo Gaurav Singh Jat
      Gaurav Singh Jat
      Blog Post Author

      Hi Enrique,

      you can send customer profiles in batch payload also from IDX with the help of "record.evaluate" and write your script to generate that. SAP CPI & Marketing cloud are able to handle that, you just need to test and adjust the batch size which CPI and Marketing can handle to achieve right performance. IDX will generate payload in array format and send defined number of customer profile in batch mode.

      Hope this will help resolve your issue.

      Author's profile photo j soler
      j soler

      Hi Gaurav Singh Jat,

      Great blog! 🙂

      Is the second part of this series, covering the CPI iflows, already published?

      Thanks and kind regards,
      Judit

      Author's profile photo Allen Chew
      Allen Chew

      Nice blog 🙂