Skip to Content

Introduction

Welcome to the blog post of the Expert Services Marketing Practice.

We are happy to share our experience with you around Marketing Integration, Analytics, and Business Technology.

Do you want to see more blogs from Expert Services? Click here

 

Table of Content

 

Overview

Integration projects should always consider the best loading data practices for better performance. In large enterprise environments, a huge amount of data is usually sent to SAP Marketing Cloud, so any change to speed up the loading process or increase response times is always welcome.

With this blog post, we will provide guidance on how to use SAP Cloud Platform Integration (CPI) features to call SAP Marketing Cloud OData API in a proper way, managing HTTP sessions in order to optimize the performance of these calls.

We will show different HTTP handling configurations, the steps to modify them in the CPI iFlow as well as highlighting the result of these session changes in SAP Marketing Cloud.

 

SAP Marketing Cloud OData Services

SAP Marketing Cloud provides a list of integration OData Services available for each entity type. The services are used to import master and transaction data into Marketing Cloud.

You may use OData services to load a huge amount of data into Marketing Cloud in order to optimize performance during these calls:

  • Reuse the http sessions (cookie and the x-CSRF token).
    • By reusing the session cookie, you avoid having additional calls to generate a new cookie every time.
    • This leads to improved performance in mass data load or landscapes with large amounts of transaction calls.

 

SAP Cloud Integration HTTP Handling Features

SAP Cloud Platform Integration (CPI) has different options to handle HTTP sessions that you can change to improve the performance in your integration scenario just re-using sessions through multiple calls.

These CPI capabilities we want to highlight with this blog to achieve the optimal performance in our interfaces.

In the CPI IFlow, choose one of the following options:

  • HTTP Session Handling “None”: Session handling is switched off
  • HTTP Session Handling “On Exchange”: Each message exchange corresponds to a single session, it is useful when your iFlow breaks down a composite message in multiple single small messages.
  • HTTP Session Handling “On Integration Flow”: Only one session will be used across the whole integration flow executions.

It is highly recommended you read the excellent blog: How to configure Session Handling in Integration Flow that describes in detail all those options in CPI.

 

Product version

For this Blog Post following product release version has been used.

SAP Marketing Cloud Release: 1805

Upgrade Schedule:

https://www.sap.com/documents/2017/01/867629d8-a27c-0010-82c7-eda71af511fa.html

SAP Cloud Platform integration Build number: 2.42.11

Upgrade Schedule:

https://blogs.sap.com/2018/06/08/sap-cloud-platform-integration-how-we-do-software-updates/

 Note:

Since we have short release cycles, the content in this blog post might look different from what you find on your SAP Marketing Cloud and SAP Cloud Platform Integration system.

  • SAP Marketing Cloud: Quarterly releases
  • SAP Cloud Platform Integration: Monthly releases

 

Integration Test Scenario

In our integration test scenario, we are going to call the public OData API Service API_MKT_CONTACT, Contact API with entity ContactOriginData through CPI iFlow to update a few master data Contacts attributes into SAP Marketing Cloud several times.

It is highly recommended you read the excellent blogs: How to import Contact Data via OData Services to SAP Marketing and How to use the new Contact OData API for SAP Marketing Cloud 1711 that describes in detail how to call OData Marketing APIs.

We created a message with multiple nodes in CPI iFlow, we will break it down to smaller single messages and call the OData API individually.

With each round of tests, we are going to change HTTP Session Handling configuration in the CPI iFlow, then demonstrate the results into SAP Marketing Cloud.

 

Development Steps

 

Incoming Message Structure

Basically, we created a XML message that contains some attributes to update the CONTACT: Contact Origin and Contact ID (keys), street name, street number, postal code and last change timestamp.

<con:UpdateContactOrigin>
  <contactsOrigin>
    <contactOrigin>
      <contactOrigin></contactOrigin>
      <contactID></contactID>
      <streetName></streetName>
      <streetNumber></streetNumber>
      <postalCode></postalCode>
      <originLastChange></originLastChange>
    </contactOrigin>
  </contactsOrigin>
</con:UpdateContactOrigin>

 

CPI iFlow

Created a simple iFlow to break down a composed message in single small messages and call SAP Marketing Cloud multiple times.

   

     

Logging Scripts was used only for this demonstration and test purposes. Use trace log level on CPI to debug and troubleshoot your iFlow tests.

IFLOW STEP DETAILS
LogPayloadBM Script step to logging inbound message before mapping step

import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;

def Message processData(Message message) {     
  def messageLog = messageLogFactory.getMessageLog(message);
  def bodyAsString = message.getBody(String.class);

  messageLog.addAttachmentAsString("InboundPayload", bodyAsString, "text/xml"); 

  return message;
}
General Splitter

Step to split a composed message in small XML packages, in our demo package equals 1.

mapAPI_UpdateContact

Simple message mapping step to map XML Incoming message structure to OData Message structure

LogPayloadAM Script step to logging outbound message after mapping step

import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;  

def Message processData(Message message) {    
  def messageLog = messageLogFactory.getMessageLog(message);
  def bodyAsString = message.getBody(String.class);

  messageLog.addAttachmentAsString("OutboundPayload", bodyAsString, "text/xml"); 

  return message;
}
OData Call

Step to define Receiver Channel

Channel Adapter: OData

Specific OData Configuration: OData Verb PUT and Batch processing

 

General Tests

 

ROUND 1: HTTP SESSION HANDLING “NONE”

 

Business Case Example: Customers would use this option in low frequency integration scenarios (Submit Sales Orders in B2B Commerce Scenarios that do not produce too many orders) or in data load that runs once a day with no splitters (Calling HTTP Services or APIs just once during runtime process).

 

In our test case, the integration scenario is receiving one huge file, it needs to be split into small single files, therefore, calling HTTP API service multiple times.

 

Using any Rest Client to send http requests (we used SOAPui for this example), we created a XML message containing 5 nodes, the iFlow will break it down in 5 singles messages then make multiple SAP Marketing API calls:

 

Single outbound message example generated after mapping iFlow step

<?xml version="1.0" encoding="UTF-8"?>
<batchParts>
  <batchChangeSet>
    <batchChangeSetPart>
      <method>PUT</method>
      <ContactOriginData>
        <ContactOriginData>
          <ContactID>testebr@sapcx.com</ContactID>
          <ContactOrigin>SAP_HYBRIS_CONSUMER</ContactOrigin>
          <ContactPostalCode>05374</ContactPostalCode>
          <AddressHouseNumber>1</AddressHouseNumber>
          <LastChangeDateTime>2018-07-17T12:14:14</LastChangeDateTime>
          <OriginDataLastChgUTCDateTime>2018-07-17T12:14:14</OriginDataLastChgUTCDateTime>
          <StreetName>Avenue TEST1</StreetName>
        </ContactOriginData>
      </ContactOriginData>
    </batchChangeSetPart>
  </batchChangeSet>
</batchParts>

 

In SAP Marketing Cloud, we can find the message data call in Import Monitor app

In this test we defined HTTP Session handling as NONE

 

First call, we sent 5 records to SAP Marketing Cloud, the iFlow called OData API 5 times that resulted in 5 HTTP Sessions opened.

 

Second call, we sent again 5 records to SAP Marketing Cloud, the iFlow called Odata API 5 more times that resulted in 10 HTTP Sessions opened.

Considerations:

Complex integration scenarios that execute several API calls during a very short time may result in many opened sessions in the backend. The system may start to refuse connections due to a possible overhead situation.

 

ROUND 2: HTTP SESSION HANDLING “ON EXCHANGE”

 

Business Case Example: Customers loading daily single batch files from SFTP to SAP Marketing Cloud (Customer payments Data load), however it is often needed to split the message in multiple composite messages (Typical for API services with restricted request size or limitations in the receiver side).

 

In this test we defined HTTP Session handling as on Exchange.

 

First call, we sent 5 records to SAP Marketing Cloud, the iFlow called OData API 5 times that resulted in only 1 HTTP Session opened.

 

Second call, we sent again 5 records to SAP Marketing Cloud, the iFlow called Odata API 5 more times that resulted in 2 HTTP Sessions opened.

 

Considerations:

Integration scenarios running frequently save a lot of http session using “on Exchange” mode, improving performance during a data load, however scenarios with multiple instances during a very short time and a high throughput may result in many opened sessions in the backend also. The system may start to refuse connections due to a possible overhead situation.

 

ROUND 3: HTTP SESSION HANDLING “ON INTEGRATION FLOW”

 

Business Case Example: Customer would use this option in B2C Commerce scenarios with multiple interactions in real time that should be replicated to SAP Marketing Cloud. This is a typical high frequency scenario interface generating thousands of calls in the receiver side.

 

In this test we defined HTTP Session handling as on Integration Flow

 

First call, we sent 5 records to SAP Marketing Cloud, the iFlow called OData API 5 times that resulted in only 1 HTTP Session opened.

 

Second call, we sent again 5 records to SAP Marketing Cloud, the iFlow called Odata API 5 more times that reused the same HTTP Session previously opened.

 

Considerations:

Integration scenarios running frequently save a lot of http session using “on Integration Flow” mode, improving performance during a data load, however make sure the shared session does not mess up your integration context scenario in case your iFlow have any global parameters that would be shared in runtime.

 

References

CPI

BLOG Cloud Integration – How to configure Session Handling in Integration Flow

BLOG Cloud Integration – Handling CSRF tokens in SAP Cloud Platform Integration

HELP Cloud Integration – Specify the Runtime Configuration

 

SAP Marketing Cloud

BLOG SAP Marketing Cloud – How to use the new Contact OData API for SAP Marketing Cloud 1711

BLOG SAP Marketing Cloud – to import Contact Data via OData Services to SAP Marketing

HELP SAP Marketing Cloud – Optimizing Performance During OData Service Calls

 

Summary

In summary of your tests. we had 2 iFlow message executions calling the SAP Marketing Cloud 10 times:

iFlow Method Sessions opened
None 10 sessions
On Exchange 2 sessions
On Integration Flow 1 reused session

 

In Productive scenarios, optimizing your integration using a suitable configuration may highly reduce concurrent sessions.  (E.g. from 50.000 parallel sessions to less than 5.000).

Customers have complex integration scenarios, data is coming from multiple origins going to SAP Marketing Cloud: Inbound contacts and product data load, marketing permissions updates, sales interaction in real time, bounces, commerce clickstream events all the time, product and offer recommendations calls from a different channel. This increased workload may slow down server performance and decrease also response time of calls. Properly handling HTTP Sessions and API calls through standard CPI features, you easily improve your data exchange besides avoiding possible overhead.

 

Make sure to read the documentation, following the guidelines and best integration practices recommended for SAP Marketing Cloud.

 

Guidance and references: SAP Marketing Cloud Help.

 

If you are still facing issues, you can always reach out to our Product Experts on the SAP Community.

 

Do you want to see more blogs from Expert Services? Click here!

To report this post you need to login first.

4 Comments

You must be Logged on to comment or reply to a post.

Leave a Reply