Automatic Outlier Detection and Event Generation i...

former_member190993 · ‎02-05-2016

Executive Summary

Event represents occurrence of an occasion in the real world that can have significant effect on the demand history. Identifying these events and removing their effect from demand history is essential to achieve good quality forecast. In enhancement package 2 of SAP SCM 7.0 demand influencing systematic and unsystematic events can be modeled. These events are based on event types that describe their meaning, rule of recurrence and length in a specified bucket size.

Promotions and Event Planning has the following business benefits:

In many industries especially consumer goods, promotions and events have a huge impact on sales volumes. This functionality helps in adjusting the sales history to obtain non-promoted historical data for the baseline forecast.
It provides the business ability to determine forecast with or without promotions/events effect.

Functional Overview

There is an existing functionality of promotion planning in APO DP to plan promotions. The event generation functionality differs with the promotion planning functionality in the following way:

In the existing promotion planning functionality with in APO DP, you create promotions in absolute quantity or in percentage (e.g. % of forecast) manually in the planning book. You choose the characteristics for the planned promotion and assign them to the promotions. The promotions for future periods can be recreated using promotion patterns that occurred in the past. However, in this functionality the input that is stored in the promotion key figure is all manual whether defined in absolute or percentages.

The historical data can also be corrected considering past promotions and outlier correction based on settings in the forecast profile. However, this correction is based on the manual quantities stored in the promotions.

On the other hand, the automatic event generation functionality explained in this whitepaper, enables the system to automatically detect the outliers and variations from the historical data based on the event type concept. In the event type you describe its meaning, rule of occurrence, length and periodicity. System then automatically identifies the impact of events on historical data based on its definition and the configuration settings for outlier detection and threshold percentages explained in this paper. The system can also estimate future events based on the identified historical events.

Unlike existing promotion planning, the price change events generation can also be modeled using this functionality wherein the short-term effect of price change on demand can be estimated by the system.

Process Description and Flow Diagram

Event types define the concept of an event. For example, if the Thanksgiving 2015 is an event, the event type is Thanksgiving. The event types have the periodicity (day, week or month) and duration defined. You can also specify the periods before and after occurrence of the actual event where the demand is impacted.

The event types can also be reoccurring; you can define the event type reoccurrence based on the following:

A simple recurrence definition (for example, the 15th of every second month)
A holiday from a holiday calendar (for example, Christmas) - This is relevant for moving events e.g. Easter, which can shift depending on the year.

Event types can be assigned to event catalogs. Event catalogs are basically the hierarchical groupings of event types.

For example:

US Public Holidays (Event Catalog)

Thanksgiving (Event Type)
Christmas (Event Type)

The planning processes that can be configured to consider event types and event catalogs are univariate forecasting, outlier detection, and interactive demand planning. The short term effect of price changes on the demand can also be modeled and can be used to estimate effect of planned price changes in the future.

The below process flow diagram shows the demand planning process. The event generation functionality influences the processes highlighted in red boxes.

1. Clean Historical Data: In this process step, the events are identified from the historical data using event generation functionality.

2. Generate Baseline Forecast: Baseline forecast is generated based on the cleansed history from point 1.

3. Forecast Adjustments based on Promotions: Future events can be estimated which can be used for forecast adjustments.

Design Consideration & Configuration

The planning processes that can be configured to consider event types and event catalogs are univariate forecasting, outlier detection, and interactive demand planning. To use the automatic event functionality, business function SCM_APO_FORECASTING_1 (SCM-APO-FCS, Events and Outliers, Causals, ABC/XYZ Classification) must be activated.

After activating the above business function, additional t-codes related to events functionality are available in the SAP easy access menu:

The below diagrams show the configuration steps needed for setting up Automatic Outlier Detection and Event Generation and Price Event Generation functionalities:

The below diagrams show the event history key figure in demand planning book where the automatically identified events are written to cleanse the raw sales history.

To define, edit and delete events, event types and event catalogs:

Advanced Planning and Optimization > Demand Planning > Environment > Demand Planning Worklist

Event types are assigned to event catalogs for hierarchical groupings:

To configure how you want the system to detect events and estimate future events, follow the below menu path in customizing-

Advanced Planning and Optimization > Supply Chain Planning > Demand Planning (DP) > Basic Settings > Events and Outliers > Configure Outlier Detection and Event Catalog Assignments

Here you can configure the settings as shown in the figures below:

Within outlier detection profiles, you define the lower and upper thresholds in percentages for assigning a specific event type to an outlier during outlier detection.

In the next node, you define your planning area and planning levels on which you want the system to detect events.

Note: You can define only one planning level for a planning area.

Planning levels are defined in a separate t-code- Advanced Planning and Optimization > Demand Planning > Environment > Current Settings > Specify Planning Levels

The below figure shows that the planning level is defined at the channel-item characteristics level.

Note: When you create a planning level, make sure that the sequence of the characteristics reflects a hierarchical relationship, from general to specific. Additionally, the promotion planning level must be the last level in the planning level definition.

Outlier Detection Periods: Here you specify the number of periods you want the system to consider before the actual period when it calculates the actual period's moving average during outlier detection.

Event catalogs are assigned to planning area in the next node:

For configuring the future event generation, you specify the event type, number of historical events and calculation method for estimating the effect of historical events on future events.

Automatic Outlier Detection and Event Generation Report

You use this report to identify the impact of different events on the historical demand. The system searches for outliers in the historical demand based on the settings defined in the customizing. After you have run the report, you can use these system identified events during forecasting to estimate the impact of future events. Additionally, you can clean the history from the impacts of events during forecasting.

Advanced Planning and Optimization > Demand Planning > Planning > Promotion > Automatic Outlier Detection and Event Generation Report

In forecasting also, you can specify that you want the system to consider the events created by the Automatic Outlier Detection and Event Generation report. This means that the system removes the effects of the events from the history that serves as the basis of forecasting.

You can also configure forecasting so that the system estimates the future occurrences of event types based on the average value of past events of the same type. On the Univariate Profile tab page, in the Event Types screen area, select the Estimate Automatically checkbox.

Pre-requisites for running the report:

Before running the automatic outlier detection report, ensure the following:

You have assigned a key figure to save the impact of events and maintained the promotion level for your planning area. Since the effects of events can be either negative or positive relative to the actual sales history, you should allow the system to store negative values in this key figure.

You have defined a planning level in customizing.

You defined settings in “Configure Outlier Detection and Event Catalog Assignments” e.g. outlier detection profiles, event catalog assignment to planning area as mentioned in section 2 above

Outliers Detection Logic

The system takes the demand history in the specified time horizon as the basis for outlier detection. If a certain combination does not have a history at the beginning or at the end of the horizon, the system adjusts the outlier detection horizon by removing the periods that do not have historical data from the beginning and from the end of the horizon.

The system calculates a moving average for the number of periods configured in Customizing, and checks if the period currently processed reaches the threshold you defined in the outlier detection profile.

The system processes the outliers by order of magnitude, which means that it first corrects the history for the period that has the largest deviation. This iteration makes sure that the outliers that would otherwise distort the moving average are handled first.

Mass Creation of Events

You use this report to create events for a specified period that is either in the future or in the past. You can create an event with the same parameters for multiple planning objects. After you have run the report, you can use the events the system has found, during forecasting to estimate the impact of future events. Additionally, you can clean the history from the impacts of events during forecasting.

If the event date you specified lies in the future, the system calculates the event's impacts based on the historic events of the same type.
If the event date you specified lies in the past, the system calculates the event's impacts from the historical data.

Example:

The event date is Nov 27, 2014 and you have defined event type in monthly buckets. In the occurrence definition of the event you define effect of the event 1 period before and 1 period after the actual event date and the actual event duration is 1 period long.

In this case, the system generates the event starting from Oct 1, 2014 which lasts for three periods.

Future Events Estimation

The system determines the impact of events in the past to estimate the events in the future. The system performs future event estimation every time you run forecasting, or manually create or delete a historical event.

The system estimates future events if the following prerequisites:

You have maintained the settings for future event estimation in Customizing
There are historical events in the system based on which the system can calculate future events
You have created events in the future horizon that the system re-estimates during future event estimation, or selected the Estimate Automatically (Estimate Future Recurring Events Automatically) checkbox in the univariate forecast profile settings.
You have specified all other settings as mentioned above in figure 1.

Price Change Events

Price change events model the short term impact of price changes on the demand. The system can estimate the price change events based on the changes in the price key figure. For example, demand might rise directly before a price increase, and drop directly after it.

In the Automatic Outlier Detection and Event Generation report, you can have the system create price change events in the past, based on historical price and demand data. The system uses the effect of the price change events in the past to estimate the short-term effect of planned price changes in the future, and uses the results in forecasting.

The following are the pre-requisites for using the price change detection functionality:

Settings in Planning Area:

Configure price settings. Define here the price key figure, price change event type and price change threshold %. For price change event type field, you should select a non-recurring event type that you have created in demand planning worklist.
The bucket you use to define the event type duration (for example, week) must match the bucket you use to detect price change events in the Automatic Outlier Detection and Event Generation report.
In the Price Change Threshold field, you specify a threshold value in percentage. The system detects price change events in periods where there is a (positive or negative) change in the price key figure compared to the previous period that is greater than the threshold value in percentage you specify.

Settings in Forecast Profile:

The bucket you specified in the Period Indicator field of the master forecast profile must match the bucket you use to detect price change events in the Automatic Outlier Detection and Event Generation report.
You must specify an MLR profile in the MLR for Univ. Fcst (MLR Profile for Univariate Forecasting) field of the univariate forecast profile. The MLR profile you specify must contain the price key figure in the MLR independent variable list.
You can specify a price change threshold in the field Price Change Threshold in the univariate forecast profile. If a value is entered here, it overrides the value set in the planning area forecast settings described above, for CVCs that have this univariate forecast profile assigned.
During the calculation of price change events, the system only applies seasonal correction in case the Periods field of the univariate forecast profile is filled, and the detection time interval you specified for the Automatic Outlier Detection and Event Generation report contains at least twice as many periods as you specified in the Periods field in the univariate forecast profile.

Deletion of Events and Event Types

You can delete events, event types, and event catalogs using any of the following ways:

From demand planning worklist
From Interactive demand planning
Using report: Delete Event Types with Dependencies
Using report: Mass Deletion of Events

Demonstration

Scenario 1: Mass Creation of Events

The figure below shows the historical data for a product in the planning book. You see that there is an increased demand during the Thanksgiving and Christmas periods in year 2014. We have created an event key figure in the planning book to store the generated events.

Program creates the 3 period long event as defined in the event type and based on the configuration settings explained above.

In the planning book, events are created for the period W51/2015, W52/2015 and W01/2016.

Events displayed with promotion ID in the planning book:

Scenario 2: Future Events Estimation

The system calculates the future events based on the historical events of the same type. You specify the event date in the future in the report for mass creation of events.

The future event calculation is based on the configuration settings done in “Configure Outlier Detection and Event Catalog Assignments” as explained above.

Scenario 3: Automatic Outlier Detection and Event Generation

In the below screenshot, the automatic event detection is executed for event type “Thanksgiving”. The event is detected and calculated by the program based on the customizing settings in “Configure Outlier Detection and Event Catalog Assignments” as explained above.

Program logs:

Event created in planning book with promotion ID:

Scenario 4: Price Change Event Generation

Automatic event generation report can also be used for detecting and generating price change events as explained above.

Settings needed for price change event detection:

You define a non-recurring price change event type and maintain price settings under forecast settings for planning area.

Create an MLR forecasting profile with price key figure as independent variable and assign this MLR profile to the univariate profile under “Combination with MLR” section.

Assign the MLR profile in the univariate profile:

Price change events are detected and generated based on the changes in the price key figure and threshold% defined in the planning area or forecast profile.

Limitations

You can use only 1 planning level per planning area. E.g. If you specify your planning level as Product-Channel-Market characteristics, you cannot use another planning level with different characteristics in the same planning area.
During forecasting, only events with the relevant bucket size are considered. E.g. When forecasting in weekly buckets, the events with weekly periodicity are only considered.
Mixed buckets are not allowed in the data view in which events are to be used.
Price change events cannot be created manually. However, these can be deleted.
In Interactive Planning or with the mass event creation report it is not possible to create historical events outside the historical horizon of the assigned forecast profile.
The characteristic for promotion level has to correspond to the lowest level of the characteristics that you use in your planning level. That is, the characteristic for promotion level has to be the last characteristic in the planning level characteristic sequence.
When the system creates multiple events that are overlapping, only one event gets the full uplift value and all the other events that overlap this event will get 0 or a very small number for the uplift value. The event identified as being the most significant will get the full uplift.

Conclusion

Ability to identify variations and outliers from historical data is essential to achieve good quality forecast. The event functionality as of SAP SCM 7.0 EHP2 can help identifying these variations based on the event type definitions. Estimation of future events can also be modeled based on the past events. The short term effect of price changes on the demand can also be modeled and can be used to estimate effect of planned price changes in the future.

Abbreviations / Acronyms

SAP	Systems, Applications, and Products in Data Processing
EhP	Enhancement Packages
APO	Advanced Planning & Optimization
DP	Demand Planning
MLR	Multiple Linear Regression

References

http://help.sap.com/SAPHELP_SCM700_EHP02/helpdata/EN/4d/33d7f2db9e00d3e10000000a42189b/frameset.htm

SAP Note: 1405656 - Implementation recommendations for SCM 7.0 APO DP

SAP Note: 1618890 - Implementation recommendations for EhP 2 of SAP SCM 7.0 DP

SCM220- Demand Planning: mySAP Supply Chain Management