Intro

PA 2.2 has been released, I have recently installed and seen a nice new feature called model compare that exists in the expert side of predictive.

Model Compare

Basically this feature tells you which algorithm has the strongest KI and KR. The algorithm that has the best KI and KR score should be the most accurate. We must remember that KI is a measurement of predictive power while KR is the measurement prediction confidence. The closer these are to 1 the stronger the model is.

Below is an example where data is run through three different algorithms, once run the algorithm with the highest KI and KR will get a star.

The results will display the best algorithm and the supporting KI and KR scores per algorithm.

You will also be able to review the results per algorithm if needed.

Hope you find this helpful.

Thanks.

Antoine CHABERTConcise & clear! Nice one Louis.

Louis de GouveiaPost authorThanks. Wonder who pointed out to me this new functionality ðŸ˜‰

Antoine CHABERTMystery ðŸ™‚

it is a mystery 10 hour version – YouTube

Cheers,

Antoine

PS/ still working on these email questions you sent me ðŸ™‚

Antoine CHABERTAlso check this nice blog entry from Hillary Bliss here: Best New Features in SAP Predictive Analytics 2.2 | SAP BI BLOG

Best regards

Antoine

Antoine CHABERTLouis De Gouveia

I realized that the model statistics and model comparison cannot be used for time series algorithms, only for classification or regression scenarios.

See our user guide for expert analytics on page 230: http://help.sap.com/businessobject/product_guides/pa22/en/pa22_expert_user_en.pdf

Model Statistics is a component that calculates performance statistics on datasets that are generated by algorithms. It can calculate statistics for two algorithm types,classification and regression.It would make sense to update the screenshots in this blog to avoid any confusion.

Sorry for the inconvenience.

Antoine

Louis de GouveiaPost authorHi Antoine

Well spotted. KI and KR is used as a measurement for the accuracy of classification and regression algorithms. Time series we should make use of MAPE to determine time series model accuracy.

I have updated the screenshots accordingly.

I look forward to future versions of SAP PA throwing an exception or warning should a logical error occur like this, or maybe even the comparison at MAPE level ðŸ˜‰

Thanks for spotting.

Antoine CHABERT+1. I raised the same ask to engineering teams 2 hours ago :-).

Antoine CHABERTFeel free to raise any product improvement idea in our idea place: Predictive Analytics: Home

I see two potential asks in this area, here we are discussing the first one mostly.

a. making the comparison of time series algorithms possible in expert – at MAPE level

b. bringing time series automated algorithms in expert analytics.

Thanks & regards

Antoine

Louis de GouveiaPost authorHi

Agree fully with point A.

Just created the idea, https://ideas.sap.com/D27885

Please vote up.

Jedson GuedesI’m new at this area and perhaps I am misunderstanding the concept of KI, but, when I tried to do an action using the SAP PA Association Rules, I got in some cases a negative KI.

How is it possible? And what does it mean?

Thanks in advance.

Antoine CHABERTHi Jedson, can you please create a dedicated question? Ideally include screenshots and the underlying dataset if possible.

Thanks & regards

Antoine

Jedson GuedesSure.

Sorry about that.

Antoine CHABERTNo problem – we just need to make sure we comply with SCN rules of engagement