Skip to Content
Product Information
Author's profile photo Yogananda Muthaiah

SAP Commissions – What I wish I had known about 🔗Smart Data Integration[SDI]

Dear All,

As you may be aware, Smart Data Integration (SDI) is seldom utilized in projects due to the availability of various ETL tools (SAP DataSphere, SAP Data Intelligence Cloud, CPI, etc.) that may be combined to move data from the source to the target system. In this manner, you may not be aware of what has to be considered, what limitations exist, how to solve your performance issues or errors or to debug, or how to go deeper for better design.

During my initial days, was not aware of whole SDI Implementation, the information was not much available where to start, how to debug/trace the errors… so started collecting all the necessary information during the projects..

Still I am by no means an expert, but over time I have compiled a list of Known things to be considered in the design that have proven extremely useful to me. If you are just starting to learn how to design and implement, this will hopefully offer you some guidance. If you are a seasoned developer, hopefully you will still learn something new.

### Note: Data Provisioning Agent version 2.3.5.2 or higher.

Available List of commands for agentcli

--createFullSystemDump   Gathers all realted log files and ini settings and creates an archive

--replicationSetup       Replication Setup Tool (Interactive operation)

--setSecureProperty      Secure Storage Utility

--configAgent            DPAgent Configuration Tool

--configAdapters         DPAgent Adapters Configuration Tool
                         Prerequisite: DPAgent should have already been started.

--configAgentIniFile     DPAgent Runtime Options (dpagent.ini editor)

--configAgentKeystore    DPAgent Keystore Configuration Utility

--LostDataTracker        Lost Data Tracker (Interactive operation)

--help                   To print this help menu

--help --all             To print detail help for all commands

<operation> --help       To print help for the given operation

How to create DP agent FullSystemDump – SDI ?

The DPAgent provides a simple utility to export all configuration, logs, and runtime environment information in a convenient ZIP format that is easy to attach in a support case.

Steps

  1. Using the command line, go to the DPAgent installation’s bin folder (Defaults are “C:\usr\sap\dataprovagent\bin” for Windows, or “/usr/sap/dataprovagent/bin” for Linux)
  2. Find agentcli.bat or agentcli.sh file, run it with the following command:
    • On Windows:
      • agentcli.bat –createFullSystemDump
    • On Linux:
      • agentcli.sh –createFullSystemDump
  3. After a few seconds of processing, the script will output the name of the ZIP file to your screen; gather and attach that file to your case.
    • Example file name and path: /usr/sap/dataprovagent/work/systemdump/<hostname>_<timestamp>_dpagent.zip

How to change “Logging Level” (Trace level) of a Data Provisioning Agent (DPAgent)?

In your DPAgent, you can enable/change the “Logging level” to trace or debug your issue.. which would be easy to detect your problems quickly.

Steps

  1. Go to DP agent installation bin folder <DPAgent_Install>\bin
  2. Run agentcli.bat –configAgent for Windows, or agentcli.sh –configAgent for Linux
  3. Choose option “Agent Preferences”
  4. Choose option “Set Agent Preferences”
  5. Choose option “Logging Level”
  6. Change level to ALL/DEBUG/TRACE/INFO according to your requirement, usually for troubleshooting problems, we need ALL level for trace
  7. No need to restart DP agent, after troubleshooting, you can follow the same steps to change trace level back to INFO

You can also manually change dpagentconfig.ini file with “framework.log.level=<NEW Value>”, this needs DP agent restart

Encryption supports for FileAdapter

You must use a version older than GnuPGP 2.3 or a version compatible with -rfc4880. For example if using Kleopatra you can use the version 3.3.16 or older: (this is specifically tested with 3.1.15 from https://gpg4win.org/change-history.html)
For 3.3.15 version, it had the following details:

GnuPGP: 2.2.27
Kleopatra: 3.1.15
GPA: 0.10.0
GpgOL: 2.4.10
GpgEX: 1.0.6
Kompendium DE: 4.0.1

How to increase DP Agent memory?

If you want to improve the performance of SDI

Steps

  1. Stop DP agent.
  2. Open the dpagent install folder.
  3. Open dpagent.ini file(e.g. /usr/sap/dataprovagent/dpagent.ini)
  4. Locate the line starting with -Xmx, to edit the value to desired size.
  5. Increase parameter -xmx to a higher value (ex: -Xmx4096m to -Xmx8192m).
  6. Save dpagent.ini
  7. Start DP agent and test again.

How to disable streaming for SDI?

SAP Support asked you to disable streaming on DP Agent – DP Server communication channel

In target HANA execute: ( SAP Commissions Support team will perform this task)

ALTER SYSTEM ALTER CONFIGURATION (‘dpserver.ini’, ‘SYSTEM’) SET (‘framework’, ‘useStreaming’) = ‘false’ WITH  RECONFIGURE;

How to update HANA XS user password for DP agent connection

Symptom

  • The HANA user used for DP agent connection with password change recently
  • User lock may happen in HANA system because of DP agent keeps retry the connection through old password
  • com.sap.hana.dp.adapter.sdk.AdapterException: Error 401 – Not authorized
  • Authentication Error

Steps

  1. At the command line, navigate to <DPAgent_root>/bin
  2. Execute one of the following commands using the –setSecureProperty parameter:
    On Windows, agentcli.bat –setSecureProperty
    On Linux, ./agentcli.sh –setSecureProperty
  3. If you want to use a different HANA User for Agent Messaging, choose Set HANA XS Username and enter the new username as prompted
  4. Choose Set HANA XS Password and enter the new password as prompted
  5. Restart the Data Provisioning Agent

How to get Remote Source Statements Status ?

It displays detailed information about executed remote queries. This information includes the query status and the number of fetched rows. This helps for monitoring the data load.

 SELECT * 
    FROM "M_REMOTE_STATEMENTS"
    ORDER BY "START_TIME";

 

I will keep adding more and more information for you which may help in your project implementation.

Assigned Tags

      Be the first to leave a comment
      You must be Logged on to comment or reply to a post.