SAP Data Intelligence 3.0 – implement with slcb tool

Last changed 18th of October, 2021

SAP Data Intelligence 3.1 – implement with slcb tool

a

The Blog – Unified Data Integration for SAP gives an Overview of the SAP Datahub 2.7.x

Overview SAP Data Intelligence 3.x

Blog – SAP Data Intelligence: Next evolution of SAP Data Hub
Blog – SAP Data Intelligence – Development News for 3.0
Blog – SAP Data Intelligence 3.0 Walk Through
Blog from the PM – How to install SAP Data Intelligence 3.0 on-premise edition

Note 2866310 – SAP Data Intelligence, on-premise edition 3.0 Release Note
Note 2820555 – SAP Data Intelligence support procedure
Note 2908055 – SAP Data Intelligence 3.0 Upgrade Note
Note 2908618 – DI 3.0 upgrade – datahub-app-database migration failed
Note 2920101 – Resolving a Prometheus “OOMKilled / no space left on device” CrashLoopBackOff in SAP Data Intelligence Diagnostics

SAP Help: New Features in SAP Data Intelligence 3.1
SAP Help: Changed Behaviors in SAP Data Intelligence 2.7 => 3.1
SAP Help: SAP Data Intelligence, on-premise Version 3.1 Patch 6 (3.1.6)
PAM: SAP Data Intelligence 3

Data Intelligence 3.0 comes in three different flavours:

click on the Picture to see the optional Installation Deployments

the Software was shipped at time Friday, 20th of March, but you will not anymore see the Software in the software Marketplace (SMP), as the distribution was moved to a repository based channel (RBSC) as other Cloud based Software as well.

Upgrade to Version 3.0.32* (9th of June 2020)

• SLPLUGIN00_49-70003322.EXE (slcb)
• MP_Stack_2000927922_2020064_SDI30_09062020.xml
• Repository 73554900100900004388

side note: the slcb screens are not differ between Version 2.x.y or 3.0.x

the slcb tool (available since SAP Data Intelligence 3.0)

Blog – How to install SAP Data Intelligence 3.0 on-premise edition

SAP Data Intelligence 3.0 (aka as SAP Datahub 2.7.x) introduces an additional option to install the images on the Hyperscaler, i.e. Azure AKS with an own Installation Pod now.
the Setup of the SAP Hostagent is not necessary anymore, and the images will be written from the SAP Repository directly into the Container Registry of the connected AKS Cluster.

Blog – prepare the Installation Host for the SLC Bridge

for the complete picture, please click on the image

Note 2589449 – Software Lifecycle Container Bridge 1.0 (aka SL Plugin) – Release Note

the Blog mentions also now the usage of the slcb tool according to kubectl or helm.

• initialize slcb to enable the container bridge
• execute slcb with stack.xml provided by the Maintenance Planer

server:~ # slcb init -u web 
server:~ # slcb execute --useStackXML /sapmnt/SDH/MP/MP_Stack_1000907257_2020034_04032020.xml -u web

when you use the new slcb Bridge the first time, you will be asked twice for the Container Registry Address: one for the slcb tool itself and secondly for the Data Intelligence Product.

When you want to upgrade the slcb tool to a higher version, then you have to re-init the slcb tool, as the individual settings are overwritten by the tool upgrade. see also the Troubleshooting section.

Please choose your slcb hostname and port wisely. You may not use the default values.

to get to the correct option, you have to choose “Advanced Installation”

please Note that the Service Type “LoadBalancer is for Hyperscaler only, while the Service Type “NotePort” is for on-premise configurations like SuSE CaaSP.

Once the slcb tool is initialized, there is also a public IP created which can be configured (DNS label) you should use the latest Kubernetes Version, e.g. 1.15.10

server:~ # kubectl get pods -n sap-slcbridge
NAME                             READY   STATUS      RESTARTS   AGE
slcbridgehead-56bbf476d5-qdgq5   2/2     Running     0          7d3h
server:~ #  kubectl -n sap-slcbridge get service
NAME                    TYPE           CLUSTER-IP    EXTERNAL-IP      PORT(S)          AGE
slcbridgehead-service   LoadBalancer   <SLC Bridge Base IP>           1128:30701/TCP   7d3h

Default slcb URL: https://<SLC Bridge Base IP>:9000/docs/index.html
Example slcb URL: https://slcb-bridge.<region>.<area>.azure.com:443/docs/index.html

Install SAP Data Intelligence 3.0 (3.0.12) or Upgrade to Patch 2 (3.0.24)

Blog – Maintenance Planer and the SLC Bridge for Data Hub

Note: the slcb tool screens are not differ between Version 2.7.x and 3.0.x, refer to the Blog – SAP DataHub 2.7 Installation with SLC Bridge

The Prerequisite Check is now reduced with the DI 3.0 Implementation

Optional screens are only opened, if they are marked

Once the Installation started, the images are copied directly to the Container Registry without temporary storing them to the Installation host.

Log Files are now accessible from the slcb UI

To ensure the eruption free usage of SAP DI 3.0 on-premise, it is suitable to increase certain memory settings before it comes to unwanted system outages.

server:/ # kubectl patch statefulsets.apps -n $NAMESPACE vsystem-vrep -p '{"spec":{"template":{"spec":{"containers":[{"name":"vsystem-vrep","resources":{"limits":{"cpu":"4", "memory":"8Gi"}}}]}}}}'
server:/ # kubectl patch deployment vsystem -n $NAMESPACE -p '{"spec": {"template": {"spec": {"containers": [{"name": "vsystem","resources":{"limits":{"memory":"2Gi"}}}]}}}}' 
server:/ # kubectl -n $NAMESPACE patch datahub default --type='json' -p='[{"op": "replace", "path": "/spec/diagnostic/resources/prometheusServer/resources", "value": {"requests":{"memory":"5G"}, "limits":{"memory":"10G"}}}]' 
server:/ # kubectl exec vsystem-vrep-0 -n $NAMESPACE -i -t -- /usr/sbin/rpc.mountd -N2 -N3 -V4 -V4.1 -V4.2 -t32

check and download the latest vctl tool version from the Launchpad before you continue.

server:/ # vctl login https://<aks-cluster>.<region>.cloudapp.azure.com system system --insecure
Enter password:
Successfully logged in as "system/system"
server:/ # vctl version
Client Version: {Version:2002.1.13-0428 BuildTime:2020-04-28T18:4725 GitCommit: Platform:linux}
Server Version: {Version:2002.1 DistributedRuntimeVersion:3.0.24 K8sVersion:v1.15.10 DeploymentType:On-prem ProductType:data-intelligence}
server:/ # 

SAP Help – Expose SAP Data Intelligence System Management Externally with Ingress

Blog – Secure the Ingress for DH 2.7 or DI 3.0

SAP Data intelligence 3.0 Logon Pad

the strategy management is redefined with two new entries mapping the latest strategy from the system and default tenant to “sdi-<tenant>-extension-strategy“.

Blog – Data Intelligence Hub – connecting the Dots …
Blog – Using new Structured Data Operators for Transformations in SAP Data Intelligence
Blog – SAP Data Intelligence Tips & Tricks

Creating Dockerfiles – For security reasons, the Modeler application has stopped supporting images that run with a root user. Non-Root Enforcement for Docker Creation

please Note the full SAP DI 3.0 Platform Version contains the Jupyter Notebook. and can be configured directly in the Application Settings of the default tenant.

Importing the combined License for SAP HANA and DATAHUB

Note 2671787 – How to request a License Keys for SAP Data Hub
Note 2823475 – System UI cannot be accessed – SAP Data Hub

Roland Kramer, SAP Platform Architect for Intelligent Data & Analytics
@RolandKramer

“I have no special talent, I am only passionately curious.”