SAP Commissions – What I wish I had known about 🔗Smart Data Integration[SDI]
As you may be aware, Smart Data Integration (SDI) is seldom utilized in projects due to the availability of various ETL tools (SAP DataSphere, SAP Data Intelligence Cloud, CPI, etc.) that may be combined to move data from the source to the target system. In this manner, you may not be aware of what has to be considered, what limitations exist, how to solve your performance issues or errors or to debug, or how to go deeper for better design.
During my initial days, was not aware of whole SDI Implementation, the information was not much available where to start, how to debug/trace the errors… so started collecting all the necessary information during the projects..
Still I am by no means an expert, but over time I have compiled a list of Known things to be considered in the design that have proven extremely useful to me. If you are just starting to learn how to design and implement, this will hopefully offer you some guidance. If you are a seasoned developer, hopefully you will still learn something new.
### Note: Data Provisioning Agent version 184.108.40.206 or higher.
Available List of commands for agentcli
--createFullSystemDump Gathers all realted log files and ini settings and creates an archive --replicationSetup Replication Setup Tool (Interactive operation) --setSecureProperty Secure Storage Utility --configAgent DPAgent Configuration Tool --configAdapters DPAgent Adapters Configuration Tool Prerequisite: DPAgent should have already been started. --configAgentIniFile DPAgent Runtime Options (dpagent.ini editor) --configAgentKeystore DPAgent Keystore Configuration Utility --LostDataTracker Lost Data Tracker (Interactive operation) --help To print this help menu --help --all To print detail help for all commands <operation> --help To print help for the given operation
How to create DP agent FullSystemDump – SDI ?
The DPAgent provides a simple utility to export all configuration, logs, and runtime environment information in a convenient ZIP format that is easy to attach in a support case.
- Using the command line, go to the DPAgent installation’s bin folder (Defaults are “C:\usr\sap\dataprovagent\bin” for Windows, or “/usr/sap/dataprovagent/bin” for Linux)
- Find agentcli.bat or agentcli.sh file, run it with the following command:
- On Windows:
- agentcli.bat –createFullSystemDump
- On Linux:
- agentcli.sh –createFullSystemDump
- On Windows:
- After a few seconds of processing, the script will output the name of the ZIP file to your screen; gather and attach that file to your case.
- Example file name and path: /usr/sap/dataprovagent/work/systemdump/<hostname>_<timestamp>_dpagent.zip
How to change “Logging Level” (Trace level) of a Data Provisioning Agent (DPAgent)?
In your DPAgent, you can enable/change the “Logging level” to trace or debug your issue.. which would be easy to detect your problems quickly.
- Go to DP agent installation bin folder <DPAgent_Install>\bin
- Run agentcli.bat –configAgent for Windows, or agentcli.sh –configAgent for Linux
- Choose option “Agent Preferences”
- Choose option “Set Agent Preferences”
- Choose option “Logging Level”
- Change level to ALL/DEBUG/TRACE/INFO according to your requirement, usually for troubleshooting problems, we need ALL level for trace
- No need to restart DP agent, after troubleshooting, you can follow the same steps to change trace level back to INFO
You can also manually change dpagentconfig.ini file with “framework.log.level=<NEW Value>”, this needs DP agent restart
Encryption supports for FileAdapter
You must use a version older than GnuPGP 2.3 or a version compatible with -rfc4880. For example if using Kleopatra you can use the version 3.3.16 or older: (this is specifically tested with 3.1.15 from https://gpg4win.org/change-history.html)
For 3.3.15 version, it had the following details:
GnuPGP: 2.2.27 Kleopatra: 3.1.15 GPA: 0.10.0 GpgOL: 2.4.10 GpgEX: 1.0.6 Kompendium DE: 4.0.1
How to increase DP Agent memory?
If you want to improve the performance of SDI
Stop DP agent.
- Open the dpagent install folder.
- Open dpagent.ini file(e.g. /usr/sap/dataprovagent/dpagent.ini)
- Locate the line starting with -Xmx, to edit the value to desired size.
- Increase parameter -xmx to a higher value (ex: -Xmx4096m to -Xmx8192m).
- Save dpagent.ini
- Start DP agent and test again.
How to disable streaming for SDI?
SAP Support asked you to disable streaming on DP Agent – DP Server communication channel
In target HANA execute: ( SAP Commissions Support team will perform this task)
ALTER SYSTEM ALTER CONFIGURATION (‘dpserver.ini’, ‘SYSTEM’) SET (‘framework’, ‘useStreaming’) = ‘false’ WITH RECONFIGURE;
How to update HANA XS user password for DP agent connection
- The HANA user used for DP agent connection with password change recently
- User lock may happen in HANA system because of DP agent keeps retry the connection through old password
- com.sap.hana.dp.adapter.sdk.AdapterException: Error 401 – Not authorized
- Authentication Error
- At the command line, navigate to <DPAgent_root>/bin
- Execute one of the following commands using the –setSecureProperty parameter:
On Windows, agentcli.bat –setSecureProperty
On Linux, ./agentcli.sh –setSecureProperty
- If you want to use a different HANA User for Agent Messaging, choose Set HANA XS Username and enter the new username as prompted
- Choose Set HANA XS Password and enter the new password as prompted
- Restart the Data Provisioning Agent
How to get Remote Source Statements Status ?
It displays detailed information about executed remote queries. This information includes the query status and the number of fetched rows. This helps for monitoring the data load.
SELECT * FROM "M_REMOTE_STATEMENTS" ORDER BY "START_TIME";