Analyzing and Predicting Electricity using SAP Lumira
Following up from Looking at my own water consumption with SAP Lumira & SAP Predictive Analytics this time I tried using SAP Lumira Desktop first. I copied my electricity usage and average daily usage from my online bill and copied that to Lumira. SAP Lumira has a predictive features but you need a time hierarchy and have to use a line chart. Your other option is to use SAP Predictive Analytics.
Here’s the sample raw data:
I bring that into SAP Lumira, create a time hierarchy on meter reading date, and then a line chart.
The gear then shows the options to forecast with SAP Predictive or Linear Regression.
I go with the SAP Predictive Analytics option,, to forecast 6 months out (how will my electric bill look at Christmas time, when the property tax bill is due?)
So you can see from the above my electricity is high in the winter (I have electric heat). I’ve looked at less inexpensive options for heat in the winter but they are not available in my area. In the summer, my electricity is low, as I don’t like having the air conditioner on (this while we’re undergoing a heat advisory today)
The forecast for December looks similar to December 2014, so on the surface the forecast looks pretty accurate.
The forecast for average daily usage for December 2015 is similar December 2014. This has allowed me to take a closer look at electricity, one of my most expensive bills.
It’s not enough (in my opinion) to enter the Data Genius contest, but maybe this will give you some ideas.
Reference
Tomorrow: Bridging the Business Analyst-Data Scientist Gap using SAP Lumira and SAP Predictive ASUG Webcast
Upcoming ASUG Business Intelligence Community W… | ASUG
Add these SAP Lumira Sessions at #SABOUC ASUG Conference in Austin
this is really cool. I wonder if you could pull the data from your utility via an API. And if so, there will soon be a hundred iPhone/Android apps to do this.
That is a great idea to ask my utility - would be nice to have this.
Hi Tammy,
As always with you, this is a great demo of the functionality of the tool. It also nicely illustrates some of the problems with the tool and a bit of the danger of SAP's drive to include predictive features without thinking really really hard about the defaults and without putting guard rails in place to protect users from mistakes. For SAP's use, here are the issues I see right off the bat:
Hopefully we'll see a real effort to include more responsible predictive features in the ... wait for it ... future.
Best,
Ethan
Actually, I should amend this. After asking someone who actually knows about this stuff, #3 isn't so much of a problem. These forecasts will usually regress to the mean. #3 is actually more of a symptom of #1 (lack of error bars), as the reason it regresses to the mean so fast is probably that there is so little data, so the prediction error bars are likely huge after the first couple of months.
I'd also suggest that application should explicitly warn the user when attempting to forecast based on insufficient data. In the case of a seasonal model, I'd think at least a few cycles would be necessary.
Hi Ethan,
Thanks a lot for your comments. I will discuss them with some SAP colleagues and we will get back to you soon.
Antoine
Hi Ethan,
Here is my point of view on your comment.
By far the biggest problem: There is no prediction interval provided for the forecast and no confidence interval for the model fit with actuals. How can a predictive tool not at the very least provide a confidence interval and at least an estimated prediction interval to give some sense of the reliability of its forecast?
I agree with you, information is lacking to the end-user to properly interpret the results he is getting.
Running the same tests using the latest version of our SAP Predictive Analytics (Automated Analytics mode) solution provides the end-user with all the missing information, especially in terms of model components and quality, estimated forecast & reliability.
Please refer to the content that I provided in appendix for more details.
The SPA forecast appears to be using the wrong auto-correlation model
(it probably shouldn't be using one at all, but rather using a pure seasonality model),
I assume, but doesn't give you options to choose a different model or parameterize the model differently.
One result of the wrong model choice is that the forecast lags the actual by about a month.
It is a core strength of our SAP Predictive Analytics product to automate the choice of the model for the end-user.
The average quality of the model that is generated in this specific example is due:
SAP Predictive Analytics (Automated Analytics mode) lets you use other input variables as well, which can improve the quality of the time series model. This is not possible in the current SAP Lumira integration.
I'd also suggest that application should explicitly warn the user when attempting to forecast based on insufficient data. In the case of a seasonal model, I'd think at least a few cycles would be necessary.
For me some potential axis to improve the current integration of predictive into Lumira should be:
As I have illustrated in my answer, all these features are covered by SAP Predictive Analytics (Automated Analytics mode).
Best regards & thank you,
Antoine
Appendix
In the appendix, I detail the results I get on the dataset that Tammy used, using our SAP Predictive Analytics 2.2 (Automated Analytics mode).
Usage (kWh)
The model generated enables only to forecast with reliability the next three months (maximal horizon). Not the next six months.
The model takes into account only 32% of the signal, as the horizon-wide MAPE is equal to 0,683.
The model is purely auto-regressive (see AR(2) mention) – no trend, no cycles detected.
Here is the confidence interval around the forecasted values. You can see the interval around the forecast is quite huge. In the table below, you see the forecasted values, err_max and err_min for the three coming months.
Meter Read Date
Usage (kWh)
Usage (kWh)_Forecasts
Err_Max
Err_Min
2015-07-19
459,99
1363,514
-443,53
2015-08-19
666,21
2240,7
-908,284
2015-09-19
779,98
2552,503
-992,541
Avg. Daily Usage
It means that the model enables only to forecast with reliability the next three months. Not the next six months.
The model takes into account only 51% of the signal.
Here is the confidence interval around the forecasted values.
You can see the interval around the forecast is quite huge.
In the table below, you see the forecasted values, err_max and err_min for the three coming months.
Note: the image formatting is not great, you have to click on each of the individual images to see their full content.
Antoine
Hi Antoine,
Thanks for the detailed response. I basically agree with everything your are saying and I appreciate you showing exactly how it works in PA as opposed to Lumira. In this case, if the Lumira integration had surfaced information about either a prediction interval (since it's so large) or the model choice (because it's non-cyclical and the user knows that this is a cyclical process), it would have been clear to the end-user that this forecast was not reliable.
I'm aware that PA is a good tool for this kind of thing, but it's also clear that the integration of predictive functionality into Lumira is severely and dangerously flawed as it currently stands.
Thanks,
Ethan
Hi Ethan,
My personal point of view is that the Lumira integration has room for improvement, in the directions I outlined in my answer:
Thanks
Antoine