SAP Commissions – What I wish I had known about 🔗Smart Data Integration[SDI]

Dear All,

As you may be aware, Smart Data Integration (SDI) is seldom utilized in projects due to the availability of various ETL tools (SAP DataSphere, SAP Data Intelligence Cloud, CPI, etc.) that may be combined to move data from the source to the target system. In this manner, you may not be aware of what has to be considered, what limitations exist, how to solve your performance issues or errors or to debug, or how to go deeper for better design.

During my initial days, was not aware of whole SDI Implementation, the information was not much available where to start, how to debug/trace the errors… so started collecting all the necessary information during the projects..

Still I am by no means an expert, but over time I have compiled a list of Known things to be considered in the design that have proven extremely useful to me. If you are just starting to learn how to design and implement, this will hopefully offer you some guidance. If you are a seasoned developer, hopefully you will still learn something new.

### Note: Data Provisioning Agent version 2.3.5.2 or higher.

Available List of commands for agentcli

--createFullSystemDump   Gathers all realted log files and ini settings and creates an archive

--replicationSetup       Replication Setup Tool (Interactive operation)

--setSecureProperty      Secure Storage Utility

--configAgent            DPAgent Configuration Tool

--configAdapters         DPAgent Adapters Configuration Tool
                         Prerequisite: DPAgent should have already been started.

--configAgentIniFile     DPAgent Runtime Options (dpagent.ini editor)

--configAgentKeystore    DPAgent Keystore Configuration Utility

--LostDataTracker        Lost Data Tracker (Interactive operation)

--help                   To print this help menu

--help --all             To print detail help for all commands

<operation> --help       To print help for the given operation

How to create DP agent FullSystemDump – SDI ?

The DPAgent provides a simple utility to export all configuration, logs, and runtime environment information in a convenient ZIP format that is easy to attach in a support case.

Steps

How to change “Logging Level” (Trace level) of a Data Provisioning Agent (DPAgent)?

In your DPAgent, you can enable/change the “Logging level” to trace or debug your issue.. which would be easy to detect your problems quickly.

Steps

Encryption supports for FileAdapter

You must use a version older than GnuPGP 2.3 or a version compatible with -rfc4880. For example if using Kleopatra you can use the version 3.3.16 or older: (this is specifically tested with 3.1.15 from https://gpg4win.org/change-history.html)
For 3.3.15 version, it had the following details:

GnuPGP: 2.2.27
Kleopatra: 3.1.15
GPA: 0.10.0
GpgOL: 2.4.10
GpgEX: 1.0.6
Kompendium DE: 4.0.1

How to increase DP Agent memory?

If you want to improve the performance of SDI

Steps

1. Stop DP agent.
2. Open the dpagent install folder.
3. Open dpagent.ini file(e.g. /usr/sap/dataprovagent/dpagent.ini)
4. Locate the line starting with -Xmx, to edit the value to desired size.
5. Increase parameter -xmx to a higher value (ex: -Xmx4096m to -Xmx8192m).
6. Save dpagent.ini
7. Start DP agent and test again.

How to disable streaming for SDI?

SAP Support asked you to disable streaming on DP Agent – DP Server communication channel

In target HANA execute: ( SAP Commissions Support team will perform this task)

ALTER SYSTEM ALTER CONFIGURATION (‘dpserver.ini’, ‘SYSTEM’) SET (‘framework’, ‘useStreaming’) = ‘false’ WITH  RECONFIGURE;

How to update HANA XS user password for DP agent connection

How to get Remote Source Statements Status ?

It displays detailed information about executed remote queries. This information includes the query status and the number of fetched rows. This helps for monitoring the data load.

 SELECT * 
    FROM "M_REMOTE_STATEMENTS"
    ORDER BY "START_TIME";

I will keep adding more and more information for you which may help in your project implementation.