SAP Data Intelligence ML Scenario Manager | Hands-on Video Tutorials

Hands-On Video Tutorials

The SAP Data Intelligence ML Scenario Manager helps you to organize your data science artifacts and to manage your tasks in a central location. As a multi-faceted data science application, it is built around the key concept of the machine learning (ML) scenario, which may contain datasets, pipelines, and Jupyter notebooks.

In this article, you will find the videos embedded with some additional information and resources. Following along in the patented zero-to-hero format, you will be ready to use SAP Data Intelligence ML Scenario Manager and work with notebooks and pipelines with minimal effort and no time wasted.

The series covers both the data scientist persona performing data exploration and free-style data science activities using Jupyter as the data engineer working with pipelines for training and interference. Postman is used to call the API to retrieve the predictions.

What You Will Learn

You can watch the 9 video tutorials in 1 hour and a bit covering how you can use SAP Data Intelligence ML Scenario Manager to create, deploy, and consume, data science projects.

What you will learn is

• Working with the Connection Manager and Semantic Data Lake connections
• Data profiling with Metadata Manager
• Working with ML Scenario Manager and Jupyter Notebooks
• Data exploration using Pandas
• Free-style data science (basic)
• How to create and execute a pipeline
• How to deploy pipelines
• How to call the API using Postman

YouTube Playlist

To bookmark or directly access the playlist, go to

• SAP Data Intelligence Machine Learning Scenario Manager

Introduction

Video Tutorial

In the first video, Tahir Hussain Babar (Bob) gives an overview of the series with references to the documentation.

Markers

0:00 – Introduction

0:30 – Scenario

2:40 – Data set

3:00 – ML Scenario Manager

3:30 – Notebook (data scientist)

4:30 – Modeler (data engineer)

5:40 – Documentation

References

For the documentation on the SAP Help Portal, see

• Machine Learning Guide

Connections and Metadata Explorer

Video Tutorial

In this video, we connect to the SAP Data Intelligence Launchpad, view connections with Connection Management and profile a data source using the Metadata Explorer. 

Markers

0:00 – Introduction

0:40 – SAP Data Intelligence Launchpad

1:15 – Connection Management

2:45 – Metadata Explorer

3:20 – Profile data set

5:45 – Monitoring

6:15 – Result

Creating Scenarios and Notebooks

Video Tutorial

The video provides a short overview ML Scenario Manager and main functionality, including working with Jupyter Notebooks.

Markers

0:00 – Introduction

1:00 – ML Scenario Manager

2:25 – Create scenario

3:50 – Create Notebook

4:15 – Working with Jupyter Notebook

Data Exploration Using Notebooks

Video Tutorial

The video shows how to connect to a dataset within our semantic data-lake within a Jupyter Notebook, and then shows how to perform basic data exploration.

Markers

0:00 – Introduction

1:00 – Create Notebook

3:30 – Install Python libraries

4:00 – Data exploration using Pandas and matplotlib

Commands

The following commands were executed

# install library 
!pip install hdfs

# read data from SDL CSV file
from hdfs import InsecureClient
import pandas as pd
client = InsecureClient('http://datalake:50070')
with client.read('/shared/scenarios/marathon/RunningTimes.csv') as reader:
  df_data = pd.read_csv(reader, delimiter=';')

# print first 5 rows of data set
df_data.head(5)

# define axis
x = df_data[["HALFMARATHON_MINUTES"]]
y_true = df_data["MARATHON_MINUTES"]

# print graph
%matplotlib inline
import matplotliv.pyplot as plot
plot.scatter(x, y_true, color = 'darkblue');
plot.xlabel("Minutes Half-Marathon");
plot.ylabel("Minutes Marathon");

Training Linear Regression Models

Video Tutorial

Scikit-learn is one of the most widely used Python packages for data science and machine learning (ML). The video shows how to perform linear regressions within a Jupyter Notebook using Scikit-learn.

Markers

0:00 – Introduction

1:00 – Scikit-learn

2:00 – Linear regression

4:00 – RMSE

5:00 – Serialise into binary format using pickle

7:30 – Create Notebooks

8:00 – Load dat file using pickle

8:15 – Make prediction

Commands

The following commands were executed in Notebook 10

# install Scikit-learn
!install sklearn

# train regression model
from sklearn.linear_model import LinearRegression
lm = LinearRegression
lm.fit(x, y_true)

# create graph with prediction
plot.scatter(x, y_true, color = 'darkblue');
plot.plot(x, lm.predict(x), color = 'red');
plot.xlabel("Actual Minutes Half-Marathon");
plot.ylabel("Actual Minutes Marathon");

# calculate RMSE
import numpy as np
y_pred = lm.predict(x)
mse = np.mean((y_pred - y_true)**2)
rmse = np.sqrt(mse)
rmse = round(rmse, 2)
print("RMSE: ", str(rmse))
print("n: ", str(len(x)))

# serialize python object into binary format
import pickle
pickle.dump(lm, open("marathon_lm.pickle.dat", "wb"))

The following commands were executed in Notebook 20

# deserialize binary 
import pickle
lm_loaded = pickle.load(open("marathon_lm.pickle.dat", "rb"))

# make prediction 
x_new = 120
predictions = lm_loaded.predict(([x_new]))
round(predictions[0], 2)

Model Training Pipeline Key Operators I

Video Tutorial

The video shows how to create graphical pipelines within the ML Scenario Manager. The initial pipeline will be used to train a model, and this video explains the key operators used in the pipeline.

Markers

0:00 – Introduction

2:20 – Create Python Producer pipeline

4:15 – Read File operator

5:25 – Python script

Model Training Pipeline Key Operators II

Video Tutorial

The video shows how to configure graphical pipelines that use Python Libraries with docker files within the ML Scenario Manager. We’ll then look at executing the pipeline.

Markers

0:00 – Introduction

1:30 – Create Docker File

2:50 – Tags

3:30 – Build

4:00 – Configure Python operator

5:00 – Pipeline overview (continued)

7:40 – Execute pipeline

8:55 – Result

Commands

The following commands were included in the Docker file

FROM $com.sap.sles.base
RUN pip3.6 install --user numpy==1.16.4
RUN pip3.6 install --user pandas==0.24.0
RUN pip3.6 install --user sklearn
RUN pip3.6 install --user scikit-learn-intelex

Rest API Inference 

Video Tutorial

The video shows how to configure graphical pipelines, built from templates, to serve REST APIs.

Markers

0:00 – Introduction

1:00 – Create Python Consumer pipeline

1:30 – Pipeline review

4:00 – Configure operator

5:50 – Tag group

6:40 – OpenAPI Server

Retrieving Predictions With Postman

Video Tutorial

The video shows how to retrieve predictions from a ML Model in SAP Data Intelligence using Postman.

Markers

0:00 – Introduction

1:30 – Deploy pipelines

3:00 – Create API request using Postman

4:50 – Authorization

5:30 – Headers

6:00 – Execute

6:20 – Recap

Learn More

SAP Community

To stay up to date, visit the topic area on the SAP Community. Follow the tag for notifications.

• SAP Data Intelligence Cloud

SAP Help Portal

For the documentation on the SAP Help Portal, see

• help.sap.com/viewer/product/SAP_DATA_INTELLIGENCE/Cloud/en-US

Jupyter

For more information about Jupyter, visit

• jupyter.org

Share and Connect

Questions? Please post as comment.

Useful? Give us a like and share on social media.

Thanks!

If you would like to receive updates, connect with me on

• LinkedIn > linkedin.com/in/dvankempen
• Twitter > @dvankempen

For the author page of SAP PRESS, visit

• Denys van Kempen