Forecasting Time Series in COVID-19 Days
Predictive Analytics and COVID-19
2020 is all about the COVID-19 pandemic, which caused the largest global recession since the Great Depression. The New York Times illustrated in its May 9 front cover the loss of more than 20 million jobs in the United States alone, in the month of April.
The COVID-19 pandemic has affected businesses and industries across the globe which is reflected in their most recent business reports and thus of course also in their underlying data.
As with Maslow–there is a hierarchy of corporate needs that companies go through when dealing with such a crisis. First companies need to respond to the crisis, then they need a clear path to recovery and eventually they need to reinvent their future.
When looked closely, the impact to each company largely varies depending on the industry where it operates and how it has been possible for this company to respond to the ongoing crisis.
Some companies faced a significant rise of their activity or were able to keep a relative stability of the business. This is the case for some internet providers, video-conferencing services, online shopping businesses and pharmaceutical companies.
Unfortunately, other companies are facing negative disruption and have limited perspectives on full recovery. This is the case for most of the aviation sector (airlines, airports, aircraft manufacturing…), the tourism industry, the arts sector (theaters, cinemas, live shows…), hotels & restaurants, the “brick and mortar” retail sector, the oil companies.
Post-crisis trajectories will vary by industry and business. Some companies will leapfrog the competition while others will be just back on-track or may be left behind. Anyway, after responding to the crisis, most companies are now entering the next phases to recover and reinvent.
In these challenging days, the ability to make fast, confident decisions is made even more crucial than ever and business intelligence and analytics platforms like SAP Analytics Cloud are here to visualize, predict and plan the evolution of business to help companies to respond, recover and reinvent.
As predictive analytics relies on past data to predict future events, the ongoing crisis also challenges the effective delivery of accurate predictions. No event in the past is comparable to what we’re experiencing today, and it’s not possible to neglect the impact to existing predictive models.
Best Practices for Time Series Forecasting Scenarios
Let’s now discuss best practices when using time series forecasting techniques to predict the evolution of key business indicators, like sales or costs. For monthly or quarterly predictive forecasts, three major scenarios can be envisioned.
In this scenario, after a few months, business resumes “as usual”, and impact is limited to a well-identified period. The assumption that the “new normal” is identical to the previous conditions must be checked thoroughly tough. The best practice consists in filtering out the months or quarters corresponding to the impacted period when training the time series models. If we assume that a company’s business gets impacted in Q2 and Q3 2020, the corresponding months or quarters will need to be removed from the underlying data foundation used to create the predictive model. In this example, Q4 2020 would be predicted based on a data history ending in Q1 2020.
In this scenario, while the activity resumes, the general behavior of the trend significantly evolves, while other components of the time series (for instance seasonal variations) remain stable. For instance, the increased work from home practices might cause new patterns for Internet consumption. End-users can form hypotheses to help estimate and predict the new evolution of the trend. These hypotheses are captured as additional variables that are part of the data foundation, not only corresponding to past events but also to future events. They introduce an element of simulation to the predictive forecasts delivered. Using such mechanisms, it can be envisioned to simulate best-case, worst-case or intermediate scenarios corresponding to the progressive reopening of the economy.
In the scenario the current and future behaviors do not reproduce any known, past patterns. Trend and seasonal variations are now totally different. The best practice there consists in discarding most of the known history and consider only the more recent data points for training the time series forecasting model. Reducing the number of data points available to train the predictive model has an impact on our ability to predict over long future periods, so expectations will likely need to be lowered and the focus should move to short-term predictive forecasts. The frequency of updates for the predictive models should be increased to cope with the rapid business evolution and possible recovery. Over time, as significant history of the new normal is being gained back, predictive models will likely regain stability, and help project for longer time horizons.
When it comes to delivering daily or weekly predictive forecasts, it will require some time to reach steady processes. A possibility is to filter out most of the history prior to the start of the pandemic period while retaining enough data points to be able to train a predictive model. Since the beginning of the data foundation used to create the predictive model might still not be totally relevant, algorithms that put more weight on recent observations should be favored.
Time Series Forecasting in SAP Analytics Cloud and SAP HANA
SAP Analytics Cloud offers automated time series forecasting capabilities. Here are the ways these capabilities can help with the scenarios described.
Using the automated time series forecasting capabilities in stories, you can test and compare the use of linear regression, triple exponential smoothing and automatic forecasting. You can also use additional inputs to help refine your predictions based on simulated evolution of the business. You can also filter the past periods that should be considered based on the relevance to generate the predictive forecasts.
Beyond the visualization aspects, you can also leverage the predictive scenarios of SAP Analytics Cloud. Predictive scenarios offers automated series forecasting capabilities with a greater degree of control and the ability to create up to thousand time series at once. Amongst the many possibilities you can select the window of observations that you use to train the time series model and filter out the data that you would consider to be irrelevant. You can also leverage additional variables as part of your data foundation (datasets) to simulate best-case or worst-case scenarios and the corresponding business evolution.
Assuming you have database development skills, you can also rely on SAP HANA predictive libraries, including for instance Triple Exponential Smoothing (available as part of the Automated Predictive Library in SAP HANA) or ARIMA and its variant Auto Arima (available as part of the Predictive Analysis Library in SAP HANA).
We are living challenging days, and despite the ongoing uncertainty, the need to analyze and to predict the business evolution is more crucial than ever. Through a combination of existing product solutions and best practices, it is possible to overcome the current challenges and continue to project towards a better future.
You can also continue your read with our detailed how-to series.