Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

SAP Data Intelligence | Hands-on Video Tutorials

dvankempen
Product and Topic Expert
Product and Topic Expert
‎02-04-2021 2:31 PM










The Digital Partner Engineering / SAP HANA Academy team just launched a new hands-on video tutorial series about SAP Data Intelligence (DI).

In this blog post you will find the videos embedded with references and additional information.

Questions? Please post as comment.

Useful? Give us a like and share on social media.

Thanks!




Hands-On Video Tutorials


What You Will Learn


Tahir Hussain Babar from Digital Partner Engineering and the SAP HANA Academy just released a new video tutorial series about the latest SAP Data Intelligence 3.1 release.

You can watch the 21-part video tutorials in a little over 2 hours. What you learn is

  • Working with Connection Management

  • Uploading data to a semantic data lake (SDL) using Metadata Explorer

  • Publishing and profiling

  • Glossaries and relationships

  • Data quality rules and dashboards

  • Data preparation and lineage

  • Working with Graphs and operators using the Modeler, including writing to files, to SAP HANA, joining data, wiretaps, validation rules, Python scripts

  • Working with Python Notebooks and the ML Scenario Manager

  • Working with REST APIs


SAP HANA Academy YouTube Playlist and Code Repository


To bookmark the playlist on YouTube, go to

For the code snippets, see




Additional Resources 


About Data Intelligence


SAP Data Intelligence is part of the Database and Data Management area of the SAP Business Technology Platform.


As advertised:

SAP Data Intelligence is a comprehensive data management solution. As the data orchestration layer of SAP’s Business Technology Platform, it transforms distributed data sprawls into vital data insights, delivering innovation at scale.



SAP Training | Training and Certification


Time of writing, there are no courses, certifications, or learning paths for SAP Data Intelligence from SAP Training.

openSAP


In January 2020, openSAP dedicated a course to SAP Data Intelligence to which you can enrol.

SAP Developer Center | Hands-On Tutorials


For the hands-on tutorials from the SAP Developer Center, visit



SAP Product Information


For product information about Data Intelligence, visit sap.com or the Discovery Center.

SAP Community


For blogs posts, questions and answers, and other community resources, visit

To be notified about the latest blog posts, follow the tag

SAP Help Portal


For the documentation, go to



Overview


In the first video, Bob gives an overview of SAP Data Intelligence and the video tutorial series. Our focus will be on the Data Intelligence Modeler, the Metadata Explorer, and Connection Management.

https://youtu.be/2tC4GBUjxuc?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction and production information from the corporate website sap.com

2:30​ - About Connection Management, Metadata Explorer, and the Modeler

4:00​ - Graphs and operators

6:00​ - APIs

6:45​ - About the documentation


Connections and Data Sources


In this video, we'll go through an overview of Connection Management. We'll then be introduced to the Metadata Explorer, and will show how to upload CSV and Parquet files into the Semantic Data Lake.

https://youtu.be/SWK7GebYjz0?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:00​ - Connection management

2:30​ - Upload data to a semantic data lake (SDL) using Metadata Explorer


Publishing and Profiling


In this video, we'll go through using the Metadata Explorer to publish data (which enables you to search the metadata, add comments to the objects, and tag datasets) as well as profile data (which helps you learn more about your data, for example, you can see if there are null or blank values, distinct and unique values, minimum and maximum and average length values).

https://youtu.be/oWbDp9CYY4w?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction 

0:40​ - Dataset metadata

1:00​ - Publication

2:30​ - Browse catalog

3:00​ - Start profiling

4:00​ - View fact sheet


Glossaries and Relationships


In this video, we'll go through using the Metadata Explorer to create business glossaries, which provide a central and shared repository for defining terms and describing how and where they are used in the business. We'll also look at the concept of relationships, which is linking terms to other terms, published datasets, rules, rulebooks, or columns.

https://youtu.be/078ysVQR95E?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:50​ - Relationships, rules, ratings and comments

2:10​ - Glossary

3:30​ - Adding terms


Data Quality Rules


In this video, we'll go through using the Metadata Explorer to create rules, which help to determine whether data complies with business constraints and requirements. We'll also look at the concept of Rulebooks.

https://youtu.be/hlPaci5joMI?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:50​ - About rules

2:00​ - Create new rule and add conditions

4:00​ - Test rule

4:30​ - About rulebooks


Data Quality Rule Dashboards


In this video, we'll go through using the Metadata Explorer to build scorecards and dashboards based upon the values of your rules. We'll also look at the concept of Rule Bindings.

https://youtu.be/zbnnIZWR--Q?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:00​ - Create new rule binding

2:00​ - Run rulebook execution

3:00​ - View results

4:00​ - Rules dashboards and add scorecard


Data Preparation


In this video, we'll go through using the Metadata Explorer to perform Self Service Data Preparation, which will involves manual cleaning, changing and joining different datasets.

https://youtu.be/3-V8qQWkHvk?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:00​ - Self-service data preparation

2:00​ - Action replace

3:00​ - Action combine columns

4:45​ - Action enrich preparation with joins

6:45​ - Action aggregation

9:00​ - Action run preparation

10:30​ - View results


Data Lineage


In this video, we'll go through using the Metadata Explorer to perform data lineage, which helps review data transformation history and metadata to quickly understand how, where, and why data has been altered.

https://youtu.be/0trolcMUoFQ?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:00​ - Start profiling

2:00​ - View lineage

3:00​ - Bind to rulebook

5:00​ - View results

5:45​ - Add to dashboard


Graphs and Operators


In this video, we'll go through using an introduction to using the Modeler to create pipelines (graphs), and also introduce two operators; the read files operator and the list files operator.

https://youtu.be/plQDlnAKodw?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:00​ - Explore data set in Metadata Explorer

3:00​ - About the Modeler, graphs, and operators

5:00​ - List Files operator

7:00​ - Read Files operator


Write Files and Message Filters


In this video, we'll go through using the Modeler to use the Write Files operators and Message Filters operators.

https://youtu.be/9QXJSprezzY?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:30​ - Write File operator

3:00​ - Message Filter and configure conversion filter

5:00​ - Add Graph Terminator


Running and Copying Graphs


In this video, we'll go through running and copying graphs when using the Modeler.

https://youtu.be/zGKLkKI9wxw?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:30​ - Running a graph

1:00​ - View results in Metadata Explorer

2:00​ - Copy graph

3:45​ - Execute graph


Writing to HANA


In this video, we'll go through using the Modeler to write to HANA.

https://youtu.be/dB3tBW83EAw?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:20​ - Operators in the Modeler

2:00​ - Workflow Trigger operator

2:30​ - Structured File Consumer operator

4:30​ - Data Transform operator: projection, aggregation, data target

8:30​ - Table Producer operator with SAP HANA Cloud connection

10:00​ - Workflow Terminator operator


Joining Data


In this video, we'll go through using the Modeler to perform some data transformation by join 2 different datasets using the Data Transform Operator.

https://youtu.be/Fo0eu5C35LY?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:00​ - Adding Workflow Trigger and Structured File Consumer operators

3:00​ - Join Data Transforms

7:00​ - Save and execute graph

7:30​ - View results in Metadata Explorer


Wiretaps


n this video, we'll go through using the Modeler to utilise Wiretaps. The Wiretap operator can wiretap a connection between two operators in a Modeler graph and display the traffic to the browser window or to an external websocket client that connects to this operator.

https://youtu.be/w0_0mIi7ohM?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:00​ - Wiretaps

1:30​ - Create new graph with Workflow Trigger operator

2:00​ - HANA Table Consumer operator

2:50​ - Flowagent CSV Producer operator

3:20​ - Wiretap operator

4:30​ - Save and run graph

5:00​ - View Wiretap output


Validation Rules


In this video, we'll go through using the Modeler to perform some data validation. Your can create rules and route records that pass through a pass output port, and also have route failed records through a fail output port.

https://youtu.be/8mMpScrA1ZU?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:45​ - Validation Rule operator

4:30​ - Save and execute

5:15​ - View wiretaps for failed ruless


Python3 Operator


In this video, we'll go through using the Modeler to run some python using the Python3 Operator.

https://youtu.be/GYtyMPVPk3Y?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:50​ - Python3 Operator with script on failed rows

2:20​ - Script  walkthrough

4:30​ - Add Wiretap operator, save and execute

For the code snippets used with the Python3 operator, see


SAP HANA Client


In this video, we'll go through using the Modeler to write data to SAP HANA Cloud using the SAP HANA Client Operator. We'll also look at all of the configuration options.

https://youtu.be/Gyv--hpav_w?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:30​ - SAP HANA Client operator 

2:10​ - Table definition in JSON forma

3:30​ - Graph Terminator operator

4:00​ - Save and execute

5:00​ - Fix errors 

6:00​ - View results in Metadata Explorer


Machine Learning Scenario Manager


In this video, we'll go through using the Machine Learning Scenario Manager (MLSM) where you can complete your data science tasks. The MLSM helps you to organize your data science artifacts and manage all tasks related to your work in one central place. An ML scenario may contain datasets, pipelines, and Jupyter Notebooks. Within the scenario, you can also manage the model performance metrics and deployment history.

https://youtu.be/9_H8CbbyytQ?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:30​ - Launch ML Scenario Manager

2:00​ - Create scenario

2:30​ - Create Notebook

3:00​ - Import Notebook

3:30​ - Code walkthrough


Using Rest API Operators


In this video, we'll go through using the Modeler to create a pipeline which uses the OpenAPI operator to expose data as an API.

https://youtu.be/a9XJ-VWmOQA?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:55​ - Create a new graph and add the OpenAPI Servlow operator

2:30​ - Add Wiretap, Write File, and Workflow Terminator

5:45​ - Save and execute


Testing Rest API Operators


In this video, we'll go through using the Modeler to test a pipeline which uses the OpenAPI operator to expose data as an API.

https://youtu.be/Fn476EqVeUc?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

0:50​ - Build a POST request using Postman adding authorization, header, and body

3:15​ - Send message and view result in wiretap and Modeler


Loading Data with Python


In this video, we'll go through using python to load data to a semantic data lake using the OpenAPI operator. We'll also look at using a sample web application to load data.

https://youtu.be/SPO0AYt0cIc?list=PLkzo92owKnVyY89xEshp_cSQ0QF8EE927

0:00​ - Introduction

1:00​ - Update and execute graph

1:30​ - Code walkthrough

3:00​ - Execute on local computer (macOS)

4:15​ - View results

5:00​ - Send message using web application (Flask)

6:00​ - View results

For the code snippets used with the Python3 operator, see


Share and Connect


Questions? Please post as comment.

Useful? Give us a like and share on social media.

Thanks!

If you would like to receive updates, connect with me on

For the author page of SAP PRESS, visit








Over the years, for the SAP HANA Academy, SAP’s Partner Innovation Lab, and à titre personnel, I have written a little over 300 posts here for the SAP Community. Some articles only reached a few readers. Others attracted quite a few more.

For your reading pleasure and convenience, here is a curated list of posts which somehow managed to pass the 10k-view mile stone and, as sign of current interest, still tickle the counters each month.

7 Comments
Labels in this area
Popular Blog Posts