You want to optimize your target groups for marketing campaigns by intelligent segmentation of your audiences? Use predictive scores based on machine learning to rank and identify the best fitting customers to increase response and conversion rates. .
In this post I describe how externally calculated score values can be pushed to SAP Marketing Cloud. The term "externally" means that the scores are modeled, trained and calculated outside SAP Marketing Cloud. Once the score values are uploaded, use the SAP Marketing Cloud infrastructure to manage and run predictive scores seamlessly for marketing use cases, like Segmentation.
The pure focus of this blog post is to describe how to setup the predictive score in SAP Marketing Cloud and how to use the Marketing Score API to fill the score with externally calculated values. All the preliminary steps like data collection and preparation, predictive model building and training, etc. is not part of this article. These topics will be addressed in a future blog. Stay tuned.
The main target persona of this blog post is the Marketing Administrator and the Data Scientist.
My thank goes to andreas.forster for setting up and running all described steps using the postman platform as well as to fcolombo and Sven Gilg for their continuous technical support on this topic.
Furthermore, there is no guarantee or support for any information or code in this blog post. Please test any content that you may want to use yourself.
Overall, a prerequisite to load external score values to SAP Marketing Cloud is to establish the communication with the system from where the scores will be uploaded from. Please see online help to do so: Communication Management - SAP Help Portal for a general overview and specifically for the Score API API for Scores - SAP Help Portal.
In short, based on the Communication Scenario SAP_COM_0307 you need to create a Communication Arrangement with Communication System and Communication User in your SAP Marketing Cloud system. A first approach to try out the Score API might then be to use postman (Postman API Platform) to check your connection and push score values to the test tenant. Later, for productive usage you can call the API directly from the Python code which runs on SAP Business Technology Platform, for example. Exactly this will be described in a technical blog post coming soon.
Before externally calculated score values can be pushed to SAP Marketing Cloud a Custom Predictive Scenario with type ‘Calculated Externally’ needs to be created and activated in the extensibility test tenant. To do so, follow these steps:
Note: You can either stay in the extensibility test system for loading external score values. Or you release the custom predictive scenario for transportation to the productive tenant and load the score values there.
More details on predictive scenarios can be found in the Extensibility Guide - Predictive Scenarios.
As mentioned in section 5 of the chapter above, a Predictive Model is needed on top of the Predictive Scenario (Score) as instance to receive the score values. Before starting to push score values call this POST request of the Score API:
The payload contains the Scenario Key (YY1_<…>) and a user-friendly name for the predictive model. For illustration we used here the example from above. The URL to call the API service should contain the addition '-api' after the system name, here an example: 'https://YOURSMC-api.s4hana.ondemand.com/'.
With the above request, a predictive model is created, activated, and persisted. Further, a Model Fit with default name "External Fit" with implementation method "Calculated Externally" is also automatically created inside the model. The generated ID of the predictive model is contained in the response of the request of the POST request.
Alternatively, the ID of the predictive model could also be retrieved with a GET request:
Instead of using a POST request to create a predictive model you could also create and activate a predictive model manually in the Predictive Studio app of SAP Marketing Cloud: Creating Predictive Models - SAP Help Portal.
As outlined above only exactly one predictive model can be created and activated per predictive scenario (score).
Note: A redundant permission check currently causes an error when trying to create a Predictive Model with the API. This will be corrected with the coming SAP Marketing Cloud release 2022. Before release 2022 is available please use the alternative to create the Predictive Model manually. I will update this blog post once the correction is out.
The generated predictive model has per default ‘0’ versions, i.e. no historical score values are stored and every time a new calculated score value with a new timestamp is uploaded via Score API the previous score value is overwritten. If previous score values shall be kept on the database for reporting reasons, the number of versions of the predictive model can be increased manually in the Predictive Studio app. For this, the user need to have the role „BUSINESS_ANALYST_MKT“ assigned. This field is labeled “No. of Days Scores are Stored”. See Saving Scores - SAP Help Portal for a detailed explanation.
You want to load the calculated score values to SAP Marketing Cloud for the prepared Predictive Scenario for a set of contacts. Example here is the above Risk Score (“YY1_CUSTOMERRISKSCORE”). For each contact you need the triple: {Timestamp | Interaction Contact UUID | Score Value}. The timestamp of all contacts needs to be the same and is typically the time when the score was calculated. It must not lay in the future and need to be later than a timestamp from a previous upload.
You can only upload score values for existing Interaction Contacts. The Score API does not create new contacts in the system. The ID of the contacts need to be known and could be extracted by CDS-Data-Extraction: Core Data Services-Based Extraction from SAP Marketing Cloud to Other SAP and Non-SAP Systems - SAP ....
If a contact ID contained in the POST request is not known to SAP Marketing Cloud the response will have a header called “sap-message” containing a XML-formatted string which gives information such as the contacts which are not found and the rejected values.
The contact ID is typically the contact UUID, but could also be the combination Object-ID and Object-Origin (see example below).
Use the following POST request to push the externally calculated score values to SAP Marketing Cloud:
The MarketingScore is the ID of the Custom Predictive Scenario, while the MarketingScoreModel is the ID of the Predictive Model created on top of the scenario.
The MarketingScoreDateTime represents be the date and time (timestamp), when the score value was calculated. The timestamp has to be the same for the score values of all contacts of the current version. This needs also to be the case, when the POST request happens in multiple packages. There is a defined version handling of score values in SAP Marketing Cloud, which is important to understand: Score Versioning - SAP Help Portal. The UTC time zone is used here.
The MarketingScoredObjectUUID is the Contact ID as UUID. Please be aware, that the UUID need to have the hyphened representation instead of the RAW16 format. This blog provides background information on Converting UUID between hyphened representation and RAW16 | SAP Blogs. In case the Contact key is not the ObjectUUID but an ObjectID with the ObjectOrigin, the POST request would look like:
The field MarketingScoreValue contains the score value as a string. The scientific format of score values is not supported.
The Custom Predictive Score is now active and filled with values. It is readily available in all Segmentation Profiles, which are based on the Segmentation Object maintained for this predictive scenario. The score can be found in the defined attribute group of the Segmentation Model. To create a segment based on the score, select the segment and then the score. In the preview area you see the distribution of the score values for the selected segment as histogram.
In the histogram select the range of values you want to keep for the sub-segment and press the button “Keep”. See Segmentation Preview for KPIs- SAP Help Portal for more details.
Note: The alternative way via the menu of the segment “Create Segment > Using an Attribute” or drag and drop the score to the segment does not work for scores.
As SAP Marketing Cloud does not offer the option to run Predictive Scenarios of the type “Calculated internally” anymore, predictive model building, training and inference need to be done outside SAP Marketing Cloud. For this, SAP Business Technology Platform (SAP BTP) offers a great infrastructure for data extraction, collection, and preparation as well as for predictive model processing. Here, the data scientist can leverage the full potential of the data to support best the business questions in focus. Lots of blog posts and articles describe the various options how to manage data and run ML on SAP BTP. Once the predictive score values are calculated they can be pushed to and seamlessly used in SAP Marketing Cloud as described in this article. I would appreciate to get your feedback and ideas around this process. Get in contact with me, when you have questions or face any issues.
In this post I describe how externally calculated score values can be pushed to SAP Marketing Cloud. The term "externally" means that the scores are modeled, trained and calculated outside SAP Marketing Cloud. Once the score values are uploaded, use the SAP Marketing Cloud infrastructure to manage and run predictive scores seamlessly for marketing use cases, like Segmentation.
The pure focus of this blog post is to describe how to setup the predictive score in SAP Marketing Cloud and how to use the Marketing Score API to fill the score with externally calculated values. All the preliminary steps like data collection and preparation, predictive model building and training, etc. is not part of this article. These topics will be addressed in a future blog. Stay tuned.
The main target persona of this blog post is the Marketing Administrator and the Data Scientist.
Table of Content
- System Communication
- Create and Activate a Custom Predictive Scenario
- Use Score API to Create and Activate a Predictive Model
- Use Score API to push external score values to a Predictive Model
- Use Custom Scores in Segmentation
- Recommendations
- Summary
My thank goes to andreas.forster for setting up and running all described steps using the postman platform as well as to fcolombo and Sven Gilg for their continuous technical support on this topic.
Furthermore, there is no guarantee or support for any information or code in this blog post. Please test any content that you may want to use yourself.
System Communication
Overall, a prerequisite to load external score values to SAP Marketing Cloud is to establish the communication with the system from where the scores will be uploaded from. Please see online help to do so: Communication Management - SAP Help Portal for a general overview and specifically for the Score API API for Scores - SAP Help Portal.
In short, based on the Communication Scenario SAP_COM_0307 you need to create a Communication Arrangement with Communication System and Communication User in your SAP Marketing Cloud system. A first approach to try out the Score API might then be to use postman (Postman API Platform) to check your connection and push score values to the test tenant. Later, for productive usage you can call the API directly from the Python code which runs on SAP Business Technology Platform, for example. Exactly this will be described in a technical blog post coming soon.
Create and Activate a Custom Predictive Scenario
Before externally calculated score values can be pushed to SAP Marketing Cloud a Custom Predictive Scenario with type ‘Calculated Externally’ needs to be created and activated in the extensibility test tenant. To do so, follow these steps:
- Login to the Extensibility test system of your client as user with the role ADMINISTRATOR_MKT.
- Enter the app ‘Predictive Scenarios’ and press the button ‘Create’. On the popup screen enter the name of the custom scenario (score), select as type ‘Calculated Externally’ and chose the value range. The values range for predictive scores should be 0 to 1 otherwise choose “Any numeric values”. Press the button ‘Create’ on the popup. The figure below illustrates this step.
- The details screen of the scenario appears, where more information needs to be maintained. Mandatory is here to enter the Segmentation Object for which the score is build and shall be used. Optionally, other metadata can be entered, like the score description, the attribute group of segmentation or the client application and visualization of the score as tile on a contact profile.
- Once all is set, save and activate the predictive scenario. Example: Do the above steps for a new custom predictive scenario with the name “Customer Risk Score”. This score gets a generated key, which is in this case “YY1_CUSTOMERRISKSCORE”. This key will be needed later when calling the Score API.
- An active scenario could now be used in the app ‘Predictive Studio’ to build a predictive model on top of it. The predictive model is basically the instance, which will be targeted, when score values are pushed via the Score API to the SAP Marketing Cloud. The next section describes, how a predictive model can be created with a POST request of the Score API. So, no manual steps are required here.
Note: You can either stay in the extensibility test system for loading external score values. Or you release the custom predictive scenario for transportation to the productive tenant and load the score values there.
More details on predictive scenarios can be found in the Extensibility Guide - Predictive Scenarios.
Use Score API to Create and Activate a Predictive Model
As mentioned in section 5 of the chapter above, a Predictive Model is needed on top of the Predictive Scenario (Score) as instance to receive the score values. Before starting to push score values call this POST request of the Score API:
POST /sap/opu/odata/sap/API_MKT_SCORE_SRV/ScoreModels
{
"MarketingScore" : "YY1_CUSTOMERRISKSCORE",
"MarketingScoreModelName" : "Customer Risk Score Model"
}The payload contains the Scenario Key (YY1_<…>) and a user-friendly name for the predictive model. For illustration we used here the example from above. The URL to call the API service should contain the addition '-api' after the system name, here an example: 'https://YOURSMC-api.s4hana.ondemand.com/'.
With the above request, a predictive model is created, activated, and persisted. Further, a Model Fit with default name "External Fit" with implementation method "Calculated Externally" is also automatically created inside the model. The generated ID of the predictive model is contained in the response of the request of the POST request.
Alternatively, the ID of the predictive model could also be retrieved with a GET request:
GET /sap/opu/odata/sap/API_MKT_SCORE_SRV/ScoreModels?$filter=(MarketingScore eq 'YY1_CUSTOMERRISKSCORE')Instead of using a POST request to create a predictive model you could also create and activate a predictive model manually in the Predictive Studio app of SAP Marketing Cloud: Creating Predictive Models - SAP Help Portal.
As outlined above only exactly one predictive model can be created and activated per predictive scenario (score).
Note: A redundant permission check currently causes an error when trying to create a Predictive Model with the API. This will be corrected with the coming SAP Marketing Cloud release 2022. Before release 2022 is available please use the alternative to create the Predictive Model manually. I will update this blog post once the correction is out.
Series of score values (Versions)
The generated predictive model has per default ‘0’ versions, i.e. no historical score values are stored and every time a new calculated score value with a new timestamp is uploaded via Score API the previous score value is overwritten. If previous score values shall be kept on the database for reporting reasons, the number of versions of the predictive model can be increased manually in the Predictive Studio app. For this, the user need to have the role „BUSINESS_ANALYST_MKT“ assigned. This field is labeled “No. of Days Scores are Stored”. See Saving Scores - SAP Help Portal for a detailed explanation.
Use Score API to push external score values to a Predictive Model
You want to load the calculated score values to SAP Marketing Cloud for the prepared Predictive Scenario for a set of contacts. Example here is the above Risk Score (“YY1_CUSTOMERRISKSCORE”). For each contact you need the triple: {Timestamp | Interaction Contact UUID | Score Value}. The timestamp of all contacts needs to be the same and is typically the time when the score was calculated. It must not lay in the future and need to be later than a timestamp from a previous upload.
You can only upload score values for existing Interaction Contacts. The Score API does not create new contacts in the system. The ID of the contacts need to be known and could be extracted by CDS-Data-Extraction: Core Data Services-Based Extraction from SAP Marketing Cloud to Other SAP and Non-SAP Systems - SAP ....
If a contact ID contained in the POST request is not known to SAP Marketing Cloud the response will have a header called “sap-message” containing a XML-formatted string which gives information such as the contacts which are not found and the rejected values.
The contact ID is typically the contact UUID, but could also be the combination Object-ID and Object-Origin (see example below).
Use the following POST request to push the externally calculated score values to SAP Marketing Cloud:
POST /sap/opu/odata/sap/API_MKT_SCORE_SRV/Scores(MarketingScore='YY1_CUSTOMERRISKSCORE')/ScoreModels
{
"MarketingScore" : "YY1_CUSTOMERRISKSCORE",
"MarketingScoreModel" : "00000000103",
"ScoreValues" : [
{
"MarketingScoreDateTime" : "2021-10-21T00:00:00",
"MarketingScoredObjectUUID" : "42010AEF-3EEC-1EDB-BC92-5F8F7D51C244",
"MarketingScoreValue" : "0.21165"
},
{
"MarketingScoreDateTime" : "2021-10-21T00:00:00",
"MarketingScoredObjectUUID" : "42010AEF-3EEC-1EDB-BC92-60B1FB9F6244",
"MarketingScoreValue" : "0.02764"
},
{
"MarketingScoreDateTime" : "2021-10-21T00:00:00",
"MarketingScoredObjectUUID" : "42010AEF-3EEC-1EDB-BC92-62F32AB52244",
"MarketingScoreValue" : "0.90456"
}
]
}
The MarketingScore is the ID of the Custom Predictive Scenario, while the MarketingScoreModel is the ID of the Predictive Model created on top of the scenario.
The MarketingScoreDateTime represents be the date and time (timestamp), when the score value was calculated. The timestamp has to be the same for the score values of all contacts of the current version. This needs also to be the case, when the POST request happens in multiple packages. There is a defined version handling of score values in SAP Marketing Cloud, which is important to understand: Score Versioning - SAP Help Portal. The UTC time zone is used here.
The MarketingScoredObjectUUID is the Contact ID as UUID. Please be aware, that the UUID need to have the hyphened representation instead of the RAW16 format. This blog provides background information on Converting UUID between hyphened representation and RAW16 | SAP Blogs. In case the Contact key is not the ObjectUUID but an ObjectID with the ObjectOrigin, the POST request would look like:
{
"MarketingScore" : "YY1_CUSTOMERRISKSCORE",
"MarketingScoreModel" : "00000000103",
"ScoreValues" : [
{
"MarketingScoreDateTime" : "2020-09-09T09:40:00",
"MarketingScoreValue" : "0.30442",
"MarketingScoredObjectID" : "1016070",
"MarketingScoredObjectOrigin" : "SAP_S4_BUPA”
}
]
}The field MarketingScoreValue contains the score value as a string. The scientific format of score values is not supported.
Use Custom Scores in Segmentation
The Custom Predictive Score is now active and filled with values. It is readily available in all Segmentation Profiles, which are based on the Segmentation Object maintained for this predictive scenario. The score can be found in the defined attribute group of the Segmentation Model. To create a segment based on the score, select the segment and then the score. In the preview area you see the distribution of the score values for the selected segment as histogram.
In the histogram select the range of values you want to keep for the sub-segment and press the button “Keep”. See Segmentation Preview for KPIs- SAP Help Portal for more details.
Note: The alternative way via the menu of the segment “Create Segment > Using an Attribute” or drag and drop the score to the segment does not work for scores.
Histogram with Score Values in Segmentation Preview.
Recommendations
- Online Help: More details, further information and examples can be found on API for Scores - SAP Help Portal.
- Do use the Score API for scores, which shall be used in Segmentation, Profiles, Lead Processing, or similar places of SAP Marketing Cloud.
- If the score is a probability score (sometimes also called ‘propensity score’), use the value range 0 to 1 for the score values.
- Do not use the Score API for loading other Key Figures like Revenue or other. Scores do not have a currency or unit and are not a percentage in SAP Marketing Cloud. For such figures other means are better suited like Custom Fields.
- High Data Volume: For high number of contacts it is recommended to call the API with packages, e.g. a size of 250.000 contacts was suitable in our environment. Evaluate an acceptable package size for your setup accordingly. Note that for multiple calls of the API in packages always use the same timestamp (version) for the score values.
- Please enter your questions regarding this topic here.
Summary
As SAP Marketing Cloud does not offer the option to run Predictive Scenarios of the type “Calculated internally” anymore, predictive model building, training and inference need to be done outside SAP Marketing Cloud. For this, SAP Business Technology Platform (SAP BTP) offers a great infrastructure for data extraction, collection, and preparation as well as for predictive model processing. Here, the data scientist can leverage the full potential of the data to support best the business questions in focus. Lots of blog posts and articles describe the various options how to manage data and run ML on SAP BTP. Once the predictive score values are calculated they can be pushed to and seamlessly used in SAP Marketing Cloud as described in this article. I would appreciate to get your feedback and ideas around this process. Get in contact with me, when you have questions or face any issues.
- SAP Managed Tags:
2 Comments
| User | Count |
|---|---|
| 4 | |
| 3 | |
| 2 | |
| 2 | |
| 2 | |
| 1 | |
| 1 | |
| 1 | |
| 1 | |
| 1 |
