Skip to Content

​Dear SAP PA Team,

This comes up as a user / consultant feedback for the SAP PA 3.2, when we tried to use it for implementing a real-time business case end to end in SAP HANA 1610 with APL / PAL installed.

Use Case: To identify invoices which can be cleared or kept open( Classification problem ).

System setup: SAP HANA 1610, APL, PAL, SAP PA 3.2

Data: Training data size 16300 records, test data 1200 records.

ETA for demo: 8 days

Our experience:

Automated models “Expert analytics” in SAP PA 3.2:

  1. So we started with the idea to train our model with SAP APL as we were less on time, hence used PA 3.2 as our base. We started with “Expert analytics” because as per documentation and help guidance, this is where we can export the automatic “model”to HANA directly with a few clicks, without much manual hassle.
  2. We built a model in “Expert analytics” and saw that the expert model of “Auto classification” gave just 60% of hit rate, without any option to tune it. Fine, we first wanted to try end to end export of atleast one model to HANA.
  3. [Feedback]: No option to tune further the “Auto classification” model ? Automated analytics running on same APL kernel has a lot of features but these are missing in “Expert” analytics.
  4. Using documentation, we connected our HANA to tool and exported the model to HANA as a procedure, we could not because it gave us an error with absolutely no explanation. For that we raised an SCN ​​post ​and feedback received was that APL and PA should be of similar version. We are already at PA 3.2, what higher can we go ? One expects that the latest version of a tool is backwards compatible.
  5. [Feedback]: PA 3.2 lacks backward compatibility for previous versions of HANA in this case. Moreover we are already on HANA 1610 so should not have been a compatibility issue !
  6. [Feedback]: The tool should be smart enough to give an error during connection step only that this version of the PA tool is not compatible with the HANA backend.

Non automated models “Expert analytics” in SAP PA 3.2:

  1. Had to revert back and built a Naive Bayes PAL model for starters, exported the model. Everything looked fine in the tool.
  2. Went to the HANA DB and were shocked to see what’s going here! When it says “Model export”, one expects a prediction function. We got a procedure that simply just queries the model’s prediction result table. Where is the prediction function ?
  3. [Feedback]: Model export feature is not exactly a model export but a results export.
  4. We tried retraining this model through SQL code as per SAP PAL documentation but that’s very time consuming, if I want to do an accuracy test and all.
  5. However, we didn’t have so much time now to try traditional algorithms so PAL was not an option to us anymore.
  6. Ok so, till here after wasting nearly 4 days. We came up to the conclusion that we have to try something else. We raised an SCN question again and started looking at the “Automated analytics” as advised to us by an expert, because we were lesser on time for our demo.

Automated analytics:

  1. Lost on hopes, we went to “Automated analytics”, created a quick classification model, exported it as HANA UDF and things looked great till here. This is the only section that worked end to end for us.
  2. [Question]: However with this feature we don’t know what is the automatic model retraining option available here, we cant keep on creating new UDFs on retrains ?
  3. Next, we thought this is a good ground to move and started building complex models, we spent nearly 2 days to tune the model and went as high as 3rd degree.
  4. Final Step: We went to “generate code” option and were shocked, there was no option to export HANA SQL for models beyond 1 degree. Why ? We raised an SCN post and came to know that the product lacks it.
  5. [Feedback]: Expert analytics needs to have option to export higher degree models as HANA UDF.
  6. Till here, we were lost on hopes and just 1 day left for demo. After spending nearly 7 days in all non-modeling tasks, we had to switch back to Automated analytics, degree 1 model using HANA UDF.
  7. Quiet an unexpected trip we had with PA 3.2.

General feedback: Overall it’s a good software to play with the data, do analytics, play with graphs( loved it ), do general analytics.

However there’s still a lot of room for improvement. It would be much appreciated, if the tool focuses more on building models which can be easily plugged in to SAP HANA as SAP customers are the first audience of this tool and they would focus more on trying to connect it with ML capabilities given in SAP HANA( PAL / APL ).

I think some of these are really important to be considered from the real project implementation perspective.

Thanks,

Hasan Rafiq

To report this post you need to login first.

3 Comments

You must be Logged on to comment or reply to a post.

  1. Michael Plazzer

    Thanks for the feedback Hasan. A few questions/comments:

    We are also trying out PA, and found that when we exported an Expert Analytics pipeline with an R component (through HANA) it would generate a procedure that queries the results table, similarly to what you found with an APL model instead of R.

    When we looked deeper, we found that when a pipeline is run in PA (with R), several views and procs are generated in HANA, which allow the model to be called again.

    Could you please check that this is not the case with APL? (I don’t have access to APL) We are putting together a little blog to describe how the process can be used, we’ll send a link through then.

    Cheers, Michael

    (0) 
    1. Hasan Rafiq Post author

      Hey Michael,

      Thanks a lot for replying on our analysis with your comments.

      As you must have read through, PA 3.2 is not able to connect with our HANA APL module due to an error message hence we have not been able to try APL. Whereas I hope this one helps:

      We tried to link PAL ( HANA – Naive Bayes ) algorithm from PA 3.2 desktop and it was exactly as you have described. In the schema it created various procedures to extract data, train and then export to another custom table. However the end procedure which is shown by the tool during export is the procedure which reads the output table.

      Our feedback was simple: An export function of the model in all ML tools is, export of the prediction reusable function. It should not be so complicated for the user to go into the system, again do all the backtracking and then extract from there relevant functions. If this much is effort is needed after using the tool, one can simply follow the flowgraph approach or the SQL code approach in HANA studio / eclipse.

      I hope this makes sense for future releases.

      Thanks,

      Hasan

      (0) 

Leave a Reply