Data driven X+O insights with Qualtrics and SAP IBP – Part 3

[Domnic]

Preparing the X – data in Qualtrics Experience Management with reference to Forecast Models in SAP Integrated Business Planning

Apart from historical sales, it is possible to understand the demand of a product by getting some idea about how customers perceive, value, trust and use a product. These factors can underpin buying decisions. They can be used to sense market demand for a product. Qualtrics Core XM helps us to capture these meaningful insights using Product satisfaction surveys which we set up in the previous blog.

In this blog, we investigate how data can be analyzed in Core XM for this specific purpose. One of the key focus is to understand how Demand Sensing algorithms in SAP Integrated Business Planning can digest this analysis from Qualtrics.

Step 1 – Understanding the target usage

We like to collect the experience data to sense demand for a product. In SAP Integrated Business Planning this is called Demand Sensing. Demand Sensing is the planning process that fine-tunes / optimizes the demand forecast at a granular weekly / daily level based on demand signals that are both internal (e.g., forecasts, sales history, shipments, open orders) and external (e.g., product satisfaction from actual users of a product). This process can accommodate a maximum of 8 external demand signals in additional to the readily available internal data. These signals can be modeled as key Figures in SAP IBP. Like every other key figure this additional signal is associated with a planning level. This level consists of 4 key attributes which are – Product, customer, location and week or day as a time axis. If we feed SAP IBP with an additional signal called Product satisfaction score from Qualtrics Core-XM, we need to insert this signal along with these 4 attributes. The survey responses already include the time stamp when the survey was done, doing an average score between the week days would give a time stamp on the score. We import the product id via the web store when the user clicked on the survey button from a product page. On the other it is also possible to capture the customer or location from the same source optionally. For this we need to define the embedded data models which was done in previous part of the blog. Hence as a next step we prepare the survey response as a numerical input (named as product satisfaction score) for machine learning algorithm used in Demand Sensing.

Firstly, we create a key figure for product satisfaction score inside the planning area in SAP IBP with a product-customer-location-week as a planning level. We need to create 3 more key figures which are needed for the demand sensing algorithm. The detailed definition is given in the below table:

Key figure	Definition	Planning Level	Business Meaning
PRODSATSCORE01	This key figure is defined in a unified planning area.  It represents the product satisfaction score from Qualtrics Core-XM. This can be used as an additional downstream / extra demand signal for Demand Sensing. This can also be reconfigured to be at daily level.	Prod-Loc-Cust-Week	Customer’s product satisfaction score
HCONVPRODSATSCORE01	HCONV helper key figure for PRODSATSCORE01. This is mandatory if this downstream / extra demand signal is added to the Demand Sensing forecast model
PRODSATSCORE01FCTADJUSTMENT	Technical DS key figure that shows short-term forecast adjustment to incorporate impact of Satisfaction score data patterns and their impact on sales.	Prod-Loc-Cust-Lag-Cal. Week	Forecast Adjustment Factor
PRODSATSCORE01MAPEIMPROV	Technical DS key figure that shows forecast error improvement projected due to short-term forecast adjustments made by DS from Satisfaction score patterns	Prod-Loc-Cust-Lag-Cal. Week	Accuracy Improvement due to Adjustment

Step 2 – Preparing the responses

In Qualtrics, we tend to keep the number of questions small and meaningful in our surveys. However, we need to interpret the survey responses into a score. We weigh the survey responses for each question from a scale of 0 to 10 depending on how meaningful they are towards perception, value, trust, usage and ultimately leading to product satisfaction.  They are available under Survey -> Tools -> Scoring. We gave different values for each answer. All the answers were a selection of multiple choices mapped with a weight.

Fig 1. Example survey question which has a score for its response

In Fig. 1, you can see a question with its multiple-choice answers and a score to each choice. You can also see that the question has an embedded data field which can be populated with a product ID. In this way, the same survey and scoring can be reused for other products or can be used to capture other metadata needed for further analysis.

Step 3 – Generate your analysis report.

For a meaning full analysis, it is recommended to collect a decent number of responses, say 500 plus. After raw data is collected in Qualtrics core-XM, we generate a report on the data based on score we did in step 2. We break this score on weekly basis as SAP IBP has the key figure defined with weekly planning level. We consider the mean values for a week as the product satisfaction score. In Core-XM it is possible to export this report as a CSV file. Doing so would allow the calculated analysis to be exported for a product.

Step 4 – Create forecast Models

In SAP IBP, we need to define what algorithms and parameters are needed to trigger a demand sensing calculation. We start by defining the key figures, which we did in step 1. As a last step, we create 2 forecast models. One considers the product satisfaction score and the other without the score to understand the influence of this additional signal in demand sensing.

1. 
   
   
   
   1. Open the “Manage Forecast Models” application in SAP IBP.
   
   
   2. Take the standard Demand Sensing model and make a copy of it using the buttons in the bottom right.
   
   
   3. Name one of them as “DemandSensingWithPSS” for demand sensing with product satisfaction score. Inside the forecasting steps define the parameters for demand sensing (full) as below.
      
      

      Fig 2. Forecast model definition considering Product satisfaction score as additional downstream signal

      
      
   
   
   4. Set the WMAPE threshold as 1%. This forces the algorithm to consider this downstream signal for optimization. The way demand sensing, especially statistical forecasting in SAP IBP works is – it learns to calculate the forecasts in multiple steps. It goes deeper and deeper considering multiple demand signals at each step. It then cross checks its learning against the consensus forecast accuracy. If the accuracy falls below this threshold, it just ignores the signal it considered. WMAPE is a standard way of measuring forecast error. Here WMAPE in Supply chain is used to tell demand sensing what is good enough to be consider as a forecast error. Getting a forecast accuracy with an error threshold of 1% is not possible, but this way we forced the algorithm to use every signal it got to improve its accuracy – it would include the additional product satisfaction score too for its forecast.
   
   
   5. Use the plus button on the right corner of Downstream Signals to add the Product satisfaction score key figure we defined in step 1.
   
   
   6. Save this and once again repeat steps a – d for a new forecast mode – just without the additional downstream signal.
   
   
   
   So far..
   
   We created a survey and defined the score for each answer in the survey response. We also exposed the survey responses as a CSV file read for SAP Integrated Business Planning. We also prepared the SAP IBP instance with respective key figures and created the forecast models that are needed for demand sensing. In the following blog, we will see how the data from Qualtrics Core-XM can be imported into SAP IBP using Cloud Platform Integration – Data Services and do some basic demand sensing using the Excel Plugin from SAP IBP.


2. 
   

   Domnic Savio Benedict

   
   

   SAP SE