SAP Data Intelligence | Hands-on Video Tutorials

The Digital Partner Engineering / SAP HANA Academy team just launched a new hands-on video tutorial series about SAP Data Intelligence (DI). In this blog post you will find the videos embedded with references and additional information. Questions? Please post as comment. Useful? Give us a like and share on social media. Thanks!

Hands-On Video Tutorials

What You Will Learn

Tahir Hussain Babar from Digital Partner Engineering and the SAP HANA Academy just released a new video tutorial series about the latest SAP Data Intelligence 3.1 release.

You can watch the 21-part video tutorials in a little over 2 hours. What you learn is

• Working with Connection Management
• Uploading data to a semantic data lake (SDL) using Metadata Explorer
• Publishing and profiling
• Glossaries and relationships
• Data quality rules and dashboards
• Data preparation and lineage
• Working with Graphs and operators using the Modeler, including writing to files, to SAP HANA, joining data, wiretaps, validation rules, Python scripts
• Working with Python Notebooks and the ML Scenario Manager
• Working with REST APIs

SAP HANA Academy YouTube Playlist and Code Repository

To bookmark the playlist on YouTube, go to

• Data Intelligence 3.1

For the code snippets, see

• github.com/saphanaacademy/di

Additional Resources 

About Data Intelligence

SAP Data Intelligence is part of the Database and Data Management area of the SAP Business Technology Platform.

As advertised:

SAP Data Intelligence is a comprehensive data management solution. As the data orchestration layer of SAP’s Business Technology Platform, it transforms distributed data sprawls into vital data insights, delivering innovation at scale.

SAP Training | Training and Certification

Time of writing, there are no courses, certifications, or learning paths for SAP Data Intelligence from SAP Training.

openSAP

In January 2020, openSAP dedicated a course to SAP Data Intelligence to which you can enrol.

• SAP Data Intelligence for Enterprise AI By Puntis Palazzolo and Matthias Sessler

SAP Developer Center | Hands-On Tutorials

For the hands-on tutorials from the SAP Developer Center, visit

• Get Started with SAP Data Intelligence, trial edition | Tutorial Mission
• SAP Data Intelligence | SAP Developers

SAP Product Information

For product information about Data Intelligence, visit sap.com or the Discovery Center.

• What is SAP Data Intelligence? | Product Information
• SAP Data Intelligence | Discovery Center

SAP Community

For blogs posts, questions and answers, and other community resources, visit

• SAP Data Intelligence | SAP Community

To be notified about the latest blog posts, follow the tag

• SAP Data Intelligence

SAP Help Portal

For the documentation, go to

• SAP Data Intelligence Cloud 
• SAP Data Intelligence (on-premises)

Overview

In the first video, Bob gives an overview of SAP Data Intelligence and the video tutorial series. Our focus will be on the Data Intelligence Modeler, the Metadata Explorer, and Connection Management.

0:00​ – Introduction and production information from the corporate website sap.com

2:30​ – About Connection Management, Metadata Explorer, and the Modeler

4:00​ – Graphs and operators

6:00​ – APIs

6:45​ – About the documentation

Connections and Data Sources

In this video, we’ll go through an overview of Connection Management. We’ll then be introduced to the Metadata Explorer, and will show how to upload CSV and Parquet files into the Semantic Data Lake.

0:00​ – Introduction

1:00​ – Connection management

2:30​ – Upload data to a semantic data lake (SDL) using Metadata Explorer

Publishing and Profiling

In this video, we’ll go through using the Metadata Explorer to publish data (which enables you to search the metadata, add comments to the objects, and tag datasets) as well as profile data (which helps you learn more about your data, for example, you can see if there are null or blank values, distinct and unique values, minimum and maximum and average length values).

0:00​ – Introduction 

0:40​ – Dataset metadata

1:00​ – Publication

2:30​ – Browse catalog

3:00​ – Start profiling

4:00​ – View fact sheet

Glossaries and Relationships

In this video, we’ll go through using the Metadata Explorer to create business glossaries, which provide a central and shared repository for defining terms and describing how and where they are used in the business. We’ll also look at the concept of relationships, which is linking terms to other terms, published datasets, rules, rulebooks, or columns.

0:00​ – Introduction

0:50​ – Relationships, rules, ratings and comments

2:10​ – Glossary

3:30​ – Adding terms

Data Quality Rules

In this video, we’ll go through using the Metadata Explorer to create rules, which help to determine whether data complies with business constraints and requirements. We’ll also look at the concept of Rulebooks.

0:00​ – Introduction

0:50​ – About rules

2:00​ – Create new rule and add conditions

4:00​ – Test rule

4:30​ – About rulebooks

Data Quality Rule Dashboards

In this video, we’ll go through using the Metadata Explorer to build scorecards and dashboards based upon the values of your rules. We’ll also look at the concept of Rule Bindings.

0:00​ – Introduction

1:00​ – Create new rule binding

2:00​ – Run rulebook execution

3:00​ – View results

4:00​ – Rules dashboards and add scorecard

Data Preparation

In this video, we’ll go through using the Metadata Explorer to perform Self Service Data Preparation, which will involves manual cleaning, changing and joining different datasets.

0:00​ – Introduction

1:00​ – Self-service data preparation

2:00​ – Action replace

3:00​ – Action combine columns

4:45​ – Action enrich preparation with joins

6:45​ – Action aggregation

9:00​ – Action run preparation

10:30​ – View results

Data Lineage

In this video, we’ll go through using the Metadata Explorer to perform data lineage, which helps review data transformation history and metadata to quickly understand how, where, and why data has been altered.

0:00​ – Introduction

1:00​ – Start profiling

2:00​ – View lineage

3:00​ – Bind to rulebook

5:00​ – View results

5:45​ – Add to dashboard

Graphs and Operators

In this video, we’ll go through using an introduction to using the Modeler to create pipelines (graphs), and also introduce two operators; the read files operator and the list files operator.

0:00​ – Introduction

1:00​ – Explore data set in Metadata Explorer

3:00​ – About the Modeler, graphs, and operators

5:00​ – List Files operator

7:00​ – Read Files operator

Write Files and Message Filters

In this video, we’ll go through using the Modeler to use the Write Files operators and Message Filters operators.

0:00​ – Introduction

0:30​ – Write File operator

3:00​ – Message Filter and configure conversion filter

5:00​ – Add Graph Terminator

Running and Copying Graphs

In this video, we’ll go through running and copying graphs when using the Modeler.

0:00​ – Introduction

0:30​ – Running a graph

1:00​ – View results in Metadata Explorer

2:00​ – Copy graph

3:45​ – Execute graph

Writing to HANA

In this video, we’ll go through using the Modeler to write to HANA.

0:00​ – Introduction

1:20​ – Operators in the Modeler

2:00​ – Workflow Trigger operator

2:30​ – Structured File Consumer operator

4:30​ – Data Transform operator: projection, aggregation, data target

8:30​ – Table Producer operator with SAP HANA Cloud connection

10:00​ – Workflow Terminator operator

Joining Data

In this video, we’ll go through using the Modeler to perform some data transformation by join 2 different datasets using the Data Transform Operator.

0:00​ – Introduction

1:00​ – Adding Workflow Trigger and Structured File Consumer operators

3:00​ – Join Data Transforms

7:00​ – Save and execute graph

7:30​ – View results in Metadata Explorer

Wiretaps

n this video, we’ll go through using the Modeler to utilise Wiretaps. The Wiretap operator can wiretap a connection between two operators in a Modeler graph and display the traffic to the browser window or to an external websocket client that connects to this operator.

0:00​ – Introduction

1:00​ – Wiretaps

1:30​ – Create new graph with Workflow Trigger operator

2:00​ – HANA Table Consumer operator

2:50​ – Flowagent CSV Producer operator

3:20​ – Wiretap operator

4:30​ – Save and run graph

5:00​ – View Wiretap output

Validation Rules

In this video, we’ll go through using the Modeler to perform some data validation. Your can create rules and route records that pass through a pass output port, and also have route failed records through a fail output port.

0:00​ – Introduction

0:45​ – Validation Rule operator

4:30​ – Save and execute

5:15​ – View wiretaps for failed ruless

Python3 Operator

In this video, we’ll go through using the Modeler to run some python using the Python3 Operator.

0:00​ – Introduction

0:50​ – Python3 Operator with script on failed rows

2:20​ – Script  walkthrough

4:30​ – Add Wiretap operator, save and execute

For the code snippets used with the Python3 operator, see

• github.com/saphanaacademy/di

SAP HANA Client

In this video, we’ll go through using the Modeler to write data to SAP HANA Cloud using the SAP HANA Client Operator. We’ll also look at all of the configuration options.

0:00​ – Introduction

0:30​ – SAP HANA Client operator 

2:10​ – Table definition in JSON forma

3:30​ – Graph Terminator operator

4:00​ – Save and execute

5:00​ – Fix errors 

6:00​ – View results in Metadata Explorer

Machine Learning Scenario Manager

In this video, we’ll go through using the Machine Learning Scenario Manager (MLSM) where you can complete your data science tasks. The MLSM helps you to organize your data science artifacts and manage all tasks related to your work in one central place. An ML scenario may contain datasets, pipelines, and Jupyter Notebooks. Within the scenario, you can also manage the model performance metrics and deployment history.

0:00​ – Introduction

1:30​ – Launch ML Scenario Manager

2:00​ – Create scenario

2:30​ – Create Notebook

3:00​ – Import Notebook

3:30​ – Code walkthrough

Using Rest API Operators

In this video, we’ll go through using the Modeler to create a pipeline which uses the OpenAPI operator to expose data as an API.

0:00​ – Introduction

0:55​ – Create a new graph and add the OpenAPI Servlow operator

2:30​ – Add Wiretap, Write File, and Workflow Terminator

5:45​ – Save and execute

Testing Rest API Operators

In this video, we’ll go through using the Modeler to test a pipeline which uses the OpenAPI operator to expose data as an API.

0:00​ – Introduction

0:50​ – Build a POST request using Postman adding authorization, header, and body

3:15​ – Send message and view result in wiretap and Modeler

Loading Data with Python

In this video, we’ll go through using python to load data to a semantic data lake using the OpenAPI operator. We’ll also look at using a sample web application to load data.

0:00​ – Introduction

1:00​ – Update and execute graph

1:30​ – Code walkthrough

3:00​ – Execute on local computer (macOS)

4:15​ – View results

5:00​ – Send message using web application (Flask)

6:00​ – View results

For the code snippets used with the Python3 operator, see

• github.com/saphanaacademy/di

Share and Connect

Questions? Please post as comment.

Useful? Give us a like and share on social media.

Thanks!

If you would like to receive updates, connect with me on

• LinkedIn > linkedin.com/in/dvankempen
• Twitter > @dvankempen

For the author page of SAP PRESS, visit

• Denys van Kempen

Over the years, for the SAP HANA Academy, SAP’s Partner Innovation Lab, and à titre personnel, I have written a little over 300 posts here for the SAP Community. Some articles only reached a few readers. Others attracted quite a few more. For your reading pleasure and convenience, here is a curated list of posts which somehow managed to pass the 10k-view mile stone and, as sign of current interest, still tickle the counters each month. Good Reads (my two cents)