SAP Data Intelligence – What’s New in DI:2107

SAP Data Intelligence, cloud edition DI:2107 is now available.

Within this blog post, you will find updates on the latest enhancements in DI:2107. We want to share and describe the new functions and features of SAP Data Intelligence for the Q3 2021 release.

If you would like to review what was made available in the previous release, please have a look at this blog post.

Overview

This section will give you a quick preview about the main developments in each topic area. All details will be described in the following sections for each individual topic area.

SAP Data Intelligence 2107

Connectivity & Integration

This topic area focuses mainly on all kinds of connection and integration capabilities which are used across the product – for example: in the Metadata Explorer or on operator level in the Pipeline Modeler.

Support HANA Calculation View with Input Parameters for Rules

Ability to run Data Quality Validation rules on SAP HANA calculation views with parameters to determine the quality of the data.

• Selection a HANA Calculation View that contains parameters as the source for rule execution within a rulebook.
• Ability to specify the parameter values to use when performing a rulebook execution against a HANA Calculation View with parameters.
• User access to parameter help that is available when entering Calculation View parameter information.

Support for database systems through Cloud Connector

Supported on-premise database systems: Oracle; MSSQL; DB2; MYSQL.

Connection types for Information Steward and Data Services are now supported via Cloud Connector as well.

Support HANA Data Lake Files as a dedicated Connection type

Introduction of a new connection type that allows to connect to HANA Data Lake Files. It is possible to:

• Write and read files using associated SAP Data Intelligence operators
• Use the new connection type in the Metadata Explorer for browsing purposes, Metadata preview and data preparation actions

Support Load Balancing for connection type ABAP_LEGACY

Connections using ABAP_LEGACY connection type now support load balancing.

Support Snowflake connection and operator

New connection type for Snowflake that can be browsed from within Metadata Explorer and provide new Data Pipeline Operators for reading data and executing statements.

Standard Template for ABAP to Data Lake scenarios

Graph templates in form of snippets that can be used to replicate data from ABAP-based systems into a data lake based on object store or Kafka.

Support of SNC encryption for RFC based ABAP connections

Support SNC encryption for connection type ABAP via RFC protocol for ABAP-based on-premise systems.

Connection Manager: show “created by” in connection list

In case of questions regarding the connection, “created by” will be the go-to person.

Metadata & Governance

In this topic area you will find all features dealing with discovering metadata, working with it and also data preparation functionalities. Sometimes you will find similar information about newly supported systems. The reason is that people only having a look into one area, do not miss information as well as there could also be some more information related to the topic area.

Scheduling Rulebooks in Metadata Explorer

USE CASE DESCRIPTION:

• Scheduling of rulebooks in Metadata Explorer
• Select to have a rulebook run on a schedule to keep your data quality information up to date
• Ability to create a rulebook and setup a schedule to automatically run the rulebook on a reoccurring basis

BUSINESS VALUE – BENEFITS:

• Always monitor the most updated rulebook results
• Automate the maintenance of rulebooks with the latest metadata
• Provide scheduling to create, modify, and view scheduled rulebooks
• View end-to-end tracking of scheduled events

SAP Information Steward on-premise Cloud Connector support for Information Steward rules (Hybrid)

USE CASE DESCRIPTION:

• Connect to an existing SAP Information Steward on-premise – browse, preview, and select projects to import rules into SAP Data Intelligence rules.
• Data Intelligence connects to on-premises SAP Information Steward instances using SAP Cloud Connector.
• Support “INFORMATION_STEWARD” connection type in SAP Data Intelligence Connection Management.
• Metadata Explorer allows users to pick an Information Steward connection to import rules and rule bindings.

BUSINESS VALUE – BENEFITS:

• Seamlessly reuse approved SAP Information Steward rules within SAP Data Intelligence.

Support Data Preparation on Failed Records

USE CASE DESCRIPTION:

• Ability to execute a rulebook and choose to have all failed rule results populated to an SAP HANA connection.
• Easily create a data preparation based a sample records that failed defined validation and quality rules.
• After creating a preparation recipe, the user will be able to apply the recipe on all the records from the source.

BUSINESS VALUE – BENEFITS:

• Increase the quality of the data
• Use data preparation to correct and improve data quality and compliance of the data.
• Effortlessly add the result of a preparation back into a rulebook to monitor quality and make sure that the results on the full dataset are improved.

Ability to support rules on SAP HANA Calc Views with parameters

USE CASE DESCRIPTION:

• Execute rules against a HANA Calculation View that contains parameters.
• Ability within a rulebook for the user to select a HANA Calculation View that contains parameters as the source for rule execution.
• Power to allow users to specify the parameter values to use when performing a rulebook execution against a HANA Calculation View with parameters.

BUSINESS VALUE – BENEFITS:

• Understand the quality of data in SAP HANA Calc Views with parameters.
• Monitor quality of SAP HANA Calc View with parameters.
• View and understand the quality trend of the HANA Calc View with parameters.

Add Profiling – Add Data Preparation – Add Publishing… within Metadata Explorer

USE CASE DESCRIPTION:

• New sources supported in Metadata Explorer (Snowflake and SAP Data Lake Files) *
• Expanded functionality support for sources **

Note: * and ** denote new with DI:2107

Pipeline Modelling

This topic area covers new operators or enhancements of existing operators. Improvements or new functionalities of the Pipeline Modeler and the development of pipelines.

Graph Debugging with Breakpoints

Upsert Mode for Structured Table Producer

Using ‘append’ write mode with the Structured Table Producer operator will surface a new property ‘upsert’ which allows the user to specify that the pipeline should overwrite records in the target based on the table primary key.

Administration

This topic area includes all services that are provided by the system – like administration, user management or system management.

Allow client certificate authentication for vsystem users

For REST API use cases, users can now request client certificates from the vSystem server when previously authenticated and use them for the duration of the configured validity duration. Admins are also able to revoke certificates or (tenant-scoped) certificate authorities. Additionally, certificate status lists can be requested to confirm the validity out-of-band.

Temporary internal user lock-out after successive erroneous password attempts

If an internal user enters incorrect password five consecutive times within a minute, the user will be locked-out temporarily for ten seconds until next attempt.

Deployment & Delivery

Within this focus area, all functions and features which are dealing with the setup process, installation or deployment will be described.

Tenant Solution Repository Isolation

For tenants using the “tenant plan”, tenant-private solution repositories are introduced. Each tenant will see its private solutions which are not visible from other tenants on the cluster. Solutions can be moved between tenants using the import/export features in the vctl command line tools or in the UI.

These are the new functions, features and enhancements in SAP Data Intelligence, cloud edition DI:2107 release.

We hope you like them and, by reading the above descriptions, have already identified some areas you would like to try out.

If you are interested, please refer to What’s New in Data Intelligence – Central Blog Post.

For more updates, follow the tag SAP Data Intelligence.

We recommend visiting our SAP Data Intelligence community topic page to check helpful resources, links and what other community members post. If you have a question, feel free to check out the Q&A area and ask a question here.

Thank you & Best Regards,

Eduardo and the SAP Data Intelligence PM team