Skip to Content
Author's profile photo Louis de Gouveia

Model Compare as a new feature in PA 2.2

Intro

PA 2.2 has been released, I have recently installed and seen a nice new feature called model compare that exists in the expert side of predictive.

Model Compare

Basically this feature tells you which algorithm has the strongest KI and KR. The algorithm that has the best KI and KR score should be the most accurate. We must remember that KI is a measurement of predictive power while KR is the measurement prediction confidence. The closer these are to 1 the stronger the model is.

Below is an example where data is run through three different algorithms, once run the algorithm with the highest KI and KR will get a star.

PA1.png

The results will display the best algorithm and the supporting KI and KR scores per algorithm.

You will also be able to review the results per algorithm if needed.

Hope you find this helpful.

Thanks.

Assigned tags

      13 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Antoine CHABERT
      Antoine CHABERT

      Concise & clear! Nice one Louis.

      Author's profile photo Louis de Gouveia
      Louis de Gouveia
      Blog Post Author

      Thanks. Wonder who pointed out to me this new functionality 😉

      Author's profile photo Antoine CHABERT
      Antoine CHABERT

      Mystery 🙂

      it is a mystery 10 hour version - YouTube

      Cheers,

      Antoine

      PS/ still working on these email questions you sent me 🙂

      Author's profile photo Antoine CHABERT
      Antoine CHABERT

      Also check this nice blog entry from Hillary Bliss here: Best New Features in SAP Predictive Analytics 2.2 | SAP BI BLOG

      Best regards

      Antoine

      Author's profile photo Antoine CHABERT
      Antoine CHABERT

      Louis De Gouveia

      I realized that the model statistics and model comparison cannot be used for time series algorithms, only for classification or regression scenarios.

      See  our user guide for expert analytics on page 230: http://help.sap.com/businessobject/product_guides/pa22/en/pa22_expert_user_en.pdf

      Model Statistics is a component that calculates performance statistics on datasets that are generated by algorithms. It can calculate statistics for two algorithm types, classification and regression.

      It would make sense to update the screenshots in this blog to avoid any confusion.

      Sorry for the inconvenience.

      Antoine

      Author's profile photo Louis de Gouveia
      Louis de Gouveia
      Blog Post Author

      Hi Antoine

      Well spotted. KI and KR is used as a measurement for the accuracy of classification and regression algorithms. Time series we should make use of MAPE to determine time series model accuracy.

      I have updated the screenshots accordingly.

      I look forward to future versions of SAP PA throwing an exception or warning should a logical error occur like this, or maybe even the comparison at MAPE level 😉

      Thanks for spotting.

      Author's profile photo Antoine CHABERT
      Antoine CHABERT

      +1. I raised the same ask to engineering teams 2 hours ago :-).

      Author's profile photo Antoine CHABERT
      Antoine CHABERT

      Feel free to raise any product improvement idea in our idea place: Predictive Analytics: Home

      I see two potential asks in this area, here we are discussing the first one mostly.

      a. making the comparison of time series algorithms possible in expert - at MAPE level

      b. bringing time series automated algorithms in expert analytics.

      Thanks & regards

      Antoine

      Author's profile photo Louis de Gouveia
      Louis de Gouveia
      Blog Post Author

      Hi

      Agree fully with point A.

      Just created the idea,  https://ideas.sap.com/D27885

      Please vote up.

      Author's profile photo Former Member
      Former Member

      I'm new at this area and perhaps I am misunderstanding the concept of KI, but, when I tried to do an action using the SAP PA Association Rules, I got in some cases a negative KI.

      How is it possible? And what does it mean?

      Thanks in advance.

      Author's profile photo Antoine CHABERT
      Antoine CHABERT

      Hi Jedson, can you please create a dedicated question? Ideally include screenshots and the underlying dataset if possible.

      Thanks & regards

      Antoine

      Author's profile photo Former Member
      Former Member

      Sure.

      Sorry about that.

      Author's profile photo Antoine CHABERT
      Antoine CHABERT

      No problem - we just need to make sure we comply with SCN rules of engagement