Last update: 12.01.2022

Machine Learning capabilities have been part of SAP HANA since the earliest version and have continuously evolved over time. These days, Machine Learning embedded in SAP HANA comes in two flavors: the Automated Predictive Library (APL) and the Predictive Analysis Library (PAL). While the automated Machine Learning of the APL targets especially developers and business analysts, the expert Machine Learning of the PAL is designed for data scientists. With the features, the contents have evolved providing hands-on tutorials, background information and updates on recent enhancements.

 

This blog post aims at providing a comprehensive overview on the most recent content. It’s meant to be a living document, so we’ll try to keep it as up to date as possible.

The links are structured as follows:

• Getting started – Your first steps with SAP HANA Machine Learning
• Machine Learning Operationalization – How to bring Machine Learning to life in real-world scenarios
• Deep-Dive sections into:
  • Classification
  • Outlier and Anomaly Detection
  • Time Series Analysis
  • Text Mining
• External Machine Learning – How to integrate with your external R or TensorFlow servers
• Miscellaneous – All other relevant materials.

Getting started

Denys van Kempen provides a broad overview on materials to get started with SAP HANA Machine Learning in this blog. You will find links to the documentation, recent demo videos as well as code samples in this collection:

• Getting Started with SAP HANA Cloud | PAL and APL

With our simple SQL interface, developers and data scientists can easily work with all the features of the SAP HANA database and integrate them with any other SQL-based solution. While SQL is considered the third most popular language for Machine Learning, we are nevertheless aware of the fact that there are other scripting languages even more popular with data scientists being specifically Python and R.

To get started with the Python and R APIs, you might want to take look at our initial release blog post, from Arun Godwin Patel:

• New R and enhanced Python API for SAP HANA Machine Learning – Released!

Arun also wrote two great articles on the power of the SAP HANA dataframe object that is introduced with the APIs. It builds the foundation of all the Machine Learning features in the library, so understanding it is important for all the following articles:

• Diving into the HANA DataFrame: Python Integration – Part 1
• Diving into the HANA DataFrame: Python Integration – Part 2

Last but not least, he introduces us to the Explanatory Data Analysis feature. This toolset specifically allows us to visually explore data using different graphics and charts, while benefiting from the remote execution of aggregation statements.

• Conquer your data with the HANA DataFrame – Exploratory Data Analysis

Jeremy Yu build on this, explaining in details how to handle Descriptive Analytics and Statistics for Data Science needs with SAP HANA Cloud in these two articles.

• Easy Descriptive Statistics with Python and SAP HANA Cloud
• Automated Descriptive Statistics Report with SAP HANA Cloud and Python

Are you looking for the ultimate step-by-step guide on how to get started with Python and Machine Learning in SAP HANA Cloud? Well, Andreas Forster has you covered with this article:

• Hands-On Tutorial: Automated Predictive (APL) in SAP HANA Cloud

Too much Python around here so far from your perspective? Perfect timing to spotlight our R API!

Yannick Shaper prepared two articles on the basics of working with R and SAP HANA, and the integration of R-based SAP HANA Machine Learning within a Shiny App:

• Hands-On Tutorial: Becoming the Chief Data Cook with RStudio and SAP HANA
• Hands-On Tutorial: Leverage SAP HANA embedded Machine Learning through an R Shiny App

After taking the first steps into Machine Learning models, you might want to bring them to life in a production scenario. But before doing that, make sure the model meets your quality criteria!

Raymond Yao has prepared a great example of how to use the new Model Report for that. It assists in understanding and debriefing a trained model by displaying model statistics, variable importance and standard metrics.

• COPD study, explanation and interpretability

 

Machine Learning Operationalization

Maintenance of the Machine Learning model lifecycle and especially versioning of different states of a model is an important part of making Machine Learning enterprise ready. This article from Xin Chen explains the details of how to set up a model storage in SAP HANA using the Python client for Machine Learning:

• Model Storage with Python Machine Learning Client

For managing and orchestration of large Data Science and Machine Learning architectures, SAP Data Intelligence comes into play. In this article Andreas Forster explains how to leverage the Python client for SAP HANA Machine Learning with the Jupyter Notebook operator in Data Intelligence:

• Hands-On Tutorial: Script and deploy Python with the new Jupyter operator in SAP Data Intelligence

To make Machine Learning part of a complex data processing workflow, you can include SAP HANA Machine Learning in your SAP Data Intelligence pipelines. The following article from Andreas gives you all the details to get started:

• SAP Data Intelligence: Deploy your first HANA ML pipelines

Your data resides in SAP Data Warehouse Cloud together with your business reports, but you don’t want to miss the power of SAP HANA Machine Learning? We’ve got you covered! Learn how to leverage SAP HANA Machine Learning from DWC:

• Leverage SAP HANA Cloud embedded Machine Learning functions in SAP Data Warehouse Cloud

Andreas Forster applies our integration with SAP DWC to bring SAP HANA Machine Learning features to the SAP Marketing Cloud:

• Extend SAP Marketing Cloud with Predictive Scores

A more detailed tutorial of how to leverage the embedded Machine Learning in SAP DATA Warehouse Cloud is provided by Christiano Hage in this article:

• Run future sales prediction using SAP HANA Cloud Machine Learning algorithms

Your scenario requires SAP HANA Machine Learning on data stored in SAP HANA, but the predictions need to be executed in an independent environment? Deploying the JavaScript export of your APL model might be an option. Learn from Andreas how to do this in this article:

• Hands-On Tutorial: Score your APL model in stand-alone JavaScript

One of the most common scenarios for SAP HANA Machine Learning is the implementation in the context of ABAP-based SAP applications, like SAP S/4HANA or SAP BW/4HANA.

Jerome Zhao showcases, how to call SAP HANA Machine Learning functions from ABAP in this article:

• An example to call PAL Apriori via AMDP

To provide a more sophisticated integration, especially with SAP S/4HANA, the Intelligent Scenario Lifecycle Management (ISLM) was introduced, to orchestrate all Machine Learning activities like the creation of scenarios and models as well as training, deployment and activation of those Machine Learning models.

Venkata Raghu Banda has prepared a comprehensive collection of materials on ISLM in this blog post:

• Resources and journey to Machine Learning with SAP S/4HANA

Some of the highlights of his collection are:

• Embedding Machine Learning into SAP S/4HANA
• Leveraging Machine Learning with the ISLM Framework
• Migration of PAi to ISLM with the embedded ML

You may also check our overview pages on Intelligent ERP and ISLM to get the latest updates:

• Intelligent ERP | SAP S/4HANA Cloud
• Intelligent Scenario Lifecycle Management for SAP S/4HANA

In case you are running an SAP BW/4HANA system and are interested in leveraging Machine Learning with the data stored in your Data Warehouse, we have you covered in this overview on how that could be done:

• Leverage SAP HANA Machine Learning with SAP BW/4HANA

 

Deep-dive sections

Let’s take a closer look into some of the most important scenarios for SAP HANA Machine Learning.

Regression

Andreas Forster created a nice demo on the use of regression techniques to predict used car prices, using the Python API.

• Hands-On Tutorial: Machine Learning push-down to SAP HANA with Python

Classification

Kurt Holst contributed a series of three blog posts focused on a classification scenario. He demonstrates the end-to-end implementation making use of the R API, as well as how to evaluate the business value of a model created that way:

• Machine Learning with SAP HANA – from R
• Machine Learning with SAP HANA – with R API Part 2
• Machine Learning with SAP HANA & R – Evaluate the Business Value

Dmitry has shared some ideas on how to derive advanced feature importance values from Hybrid Gradient Boosting Trees for classification scenarios in this post:

• HANA Cloud advanced feature importance with help of HGBDT

Outlier and Anomaly Detection

Likun Hou has prepared four blog entries on different techniques for outlier and anomaly detection.

The first article demonstrates the usage of Statistical Tests (Variance Test and IQR Test) for Outlier Detection. Likun shows that IQR test is a more robust outlier detection method with the presence of extremely deviated (from mean/median) values in the targeted numerical feature. However, both methods only work on 1-dimensional numerical data, so they are mainly applicable to outliers with at least one outstanding numerical features value.

• Outlier Detection using Statistical Tests

The second option is to use the DBSCAN clustering algorithm to perform Outlier Detection. Different from the Statistical test approach above, all feature values can get involved if appropriate distance metrics are adopted.

• Outlier Detection by Clustering

Typically, these two methods (Statistical Test and Clustering) can only detect outliers in the input dataset, and the detection result cannot be generalized to new data points, because they do not come up with any model. The third article demonstrates how Classification methods can be adopted to overcome this difficulty.

• Outlier Detection with One-class Classification

However, all the aforementioned techniques become less applicable, when the dataset of interest is of high dimensionality (i.e. contains many features), or the boundary between normal points and anomalous ones is complicated. In his fourth article, Likun demonstrates a better approach by manually labeling the point of anomalies in the dataset, and then training a supervised Machine Learning model for the classification of normal points and anomalies.

• Learning from Labeled Anomalies for Efficient Anomaly Detection

Another approach to Anomaly detection is based on sensor data over time, that requires the usage of time series analysis techniques. We have the basics of that covered in the section below. Nidhi Sawhney and Rafael Pacheco showcase two scenarios in these three articles:

• Quality Identification of rail-road tracks – An application of Dynamic Time Warping using SAP HANA PAL
• Anomaly Forecast of Sensor Data in Energy Intensive Industries – Part I: The Machine Learning and Beer Production
• Anomaly Forecast of Sensor Data in Energy Intensive Industries – Part II: The Machine Learning Execution

Andreas Forster and Nico Wang have published another article on that approach, describing how to detect contextual anomalies:

• Detecting Contextual Anomalies with SAP HANA ML

Finally, Raymond Yao shows us how to apply Weibull Analysis – one of the most used algorithms for Predictive Maintenance use cases.

• Weibull Analysis using Python Machine Learning client for SAP HANA

Time-Series Analysis

Another series of great articles from Likun Hou covers the most relevant aspects of Time-Series Analysis.

He starts off with explaining the basic principles of Time-Series Analysis, specifically the ideas of “Trends” and “Seasonality” and how to perform decomposition on these to prepare for an Anomaly Detection.

• Anomaly Detection in Time-Series using Seasonal Decomposition

The second article explains how to apply the most commonly used techniques for Time-Series Analysis: Exponential Smoothing and ARIMA.

• Time-Series Modeling and Analysis

Lastly, Likun introduces one of the most recent enhancement of the Predictive Analysis Library (PAL): The Additive Model Time-series Analysis, that is an advanced approach that proves to be superior in dealing time-series with complicated trend, multiple seasonality as well as cyclic patterns.

• Additive Model Time-series Analysis

Xin Chen dedicated another article to Seasonal Decomposition, showcasing examples of how this can be done with SAP HANA PAL.

• Identification of Seasonality in Time Series

While many Time-Series scenarios are based on just one time-dependent variable, there are also many cases where Time-Series consist of more than one time-dependent variable and each variable depends not only on its past values but also has some dependency on other variables. These Multivariate Time-Series are covered in this article:

• A Multivariate Time Series Modeling and Forecasting Guide

Marc Daniau introduces us to a recently added feature in the Automated Predictive Library, called Piecewise Linear Trend, that can be specifically helpful in detecting and handling change points in time series.

• Piecewise Linear Trend with Automated Time Series Forecasting (APL)

Text Mining

The most recent enhancement to the SAP HANA Machine Learning features is the Text Mining feature. The initial version allows for analysis and classification of texts, like service tickets or text messages and enable users to explore relations among the texts. Learn how to make use of this feature in Alex Dalentzas blog post:

• Text mining at SAP HANA Cloud with Python

Maxime Simon has prepared a more in-depth article on how to use this feature for automated classification of messages with customer complaints.

• Text Mining on SAP HANA Cloud

 

Multi-Model Data Science

The true power of SAP HANA for Data Science becomes visible when working with different data types. Since SAP HANA is a multi-model database, it supports various data types like JSON documents, graph networks or spatial data. And of course, we can use these for Machine Learning as well.

Mathias Kemeter has two great examples on how Machine Learning can be used on spatial data in these two articles.

• Spatial Data Science powered by SAP HANA
• Predicting Taxi Destinations with SAP HANA

 

External Machine Learning

The third flavor of the SAP HANA Machine Learning is the integration of external Machine Learning servers. It mainly allows us to remotely execute Machine Learning models in TensorFlow or R on separate servers using data from a SAP HANA database (on-premises) and consuming back the results in SAP HANA as well.

These two articles provide an overview of the R server integration.

• SAP HANA and R hands-on: From freestyle to deployment
• Parallelization options with the SAP HANA and R-Integration

More information on the TensorFlow integration can be found in these posts from Philip Mugglestone and Nandi Kishore:

• Introducing SAP HANA External Machine Learning (aka TensorFlow Integration)
• Tensorflow Machine Learning Model Integration with SAP HANA

 

Miscellaneous

Sometimes, you have the right data, in the right place, at the right time, but your scenario requires them to be turned by 90 degrees. That can be cumbersome, but Nidhi Sawhney shows us, how pivoting can be easily done, using SAP HANA and the Python API.

• Pivoting Data with SAP HANA

This article describes, how to import multiple excel files into a single SAP HANA table using the Python Machine Learning client for SAP HANA.

• Import multiple excel files into a single SAP HANA table

If you are looking for more code examples on the use of SAP HANA Machine Learning, please take a look at our sample repository on Github, to find dozens of examples for the various use cases of Machine Learning.

• SAP HANA Predictive and Machine Learning Scenarios

 

Thank you!

I would like to thank all the above contributors for their tremendous effort and time to create these valuable materials!

As said in the introduction, this article will receive updates any time new relevant content gets created. If there is anything you miss in this collection (either because we missed it or because there is no resources on your specific topic), do not hesitate to reach to Christoph Morgen (SAP HANA Product Management) or me (SAP HANA Solution Management).

Also, we are happy to take your feedback, thoughts or questions in the comment section below!