Skip to Content

SAP Hana EIM (SDI/SDQ) setup

In my documentation I’ll explain how to setup and configure SAP Hana SP10 EIM (SDI/SDQ) with Sybase IQ database and ERP on Hana database schema source system to replicated data for realtime data replication.

I will show in detail step and configuration point to achieve it.

In order execution

  • Create Sybase IQ database
  • Enable DP server for SDI
  • Enable Script server for SDQ
  • Install SDQ cleanse and geocode directory
  • Install DU HANA_IM_DP (Data provisioning)
  • Install and Register Data Provisioning Agent
  • Create remote source
  • Data replication and monitoring

Configuration required on SP9

The xsengine needs to be turn to true (if not done)

The statistic server needs to be turn to true (if not done)

The DU HANA_IM_ESS needs to be imported

Guide used

SAP Hana EIM Administration Guide SP10
SAP Hana EIM Configuration guide SP10

Note used

179583 – SAP HANA Enterprise Information Management SPS 10 Central Release Note

Link used

Overview Architecture

7-15-2015 6-09-06 PM.jpg

Starting Hana SP9 the new features call SDI (Smart Data Integration) and SDQ (Smart Data Quality) has been introduce.

The purpose of these new features is to leverage an integrated ETL mechanism directly into Hana over SDA

To make it simple:

  • Smart Data Integration provide data replication and transformation services
  • Smart Data Quality provide an advanced transformation to support data quality functionality

Create Sybase IQ database

In order to have a dedicated database to work with I’ll create my own database into IQ server:

From the SCC go to administration and proceed as follow
7-10-2015 3-55-36 PM.jpg

7-10-2015 4-00-54 PM.jpg

7-10-2015 4-01-28 PM.jpg

SCC agent password: the password define during the IQ server installation
Utility server password: auto fill do not change it
IQ server port: use an unused port, I already have 2 db running so I pic the next number
Database path: <path where the db is stored><dbname>.db
IQ main dbspace path: <path where the dbspace is stored><dbname>.iq
7-10-2015 4-06-10 PM.jpg

Check mark ok

7-10-2015 4-14-09 PM.jpg

7-10-2015 4-18-37 PM.jpg

7-10-2015 4-19-51 PM.jpg

7-10-2015 4-20-47 PM.jpg

My database now available I’ll create 3 simple tables for this test, I’ll use interactive SQL
7-10-2015 4-49-11 PM.jpg

With the following syntax

7-10-2015 4-53-48 PM.jpg

Enable Data Provisioning server for SDI

When Hana is installed, by default the DP server is note activate, in order to have the ability to use SDI it needs to be enabled. The value needs to be change to 1
7-10-2015 5-22-47 PM.jpg

Enable Script server for SDQ

To take advantage of the SDQ functionality the script server value needs to be change to 1
7-10-2015 5-47-54 PM.jpg

Install SDQ cleanse and geocode directory

The Cleanse and Geocode nodes rely on reference data found in directories where we download and deploy to the SAP HANA server.

To download those directories go on the SMP and select the one you need
You can download several directories depending on what you are licensed for.
7-10-2015 8-50-57 PM.jpg

7-10-2015 8-55-08 PM.jpg

Once downloaded decompress it at the following location:
7-11-2015 8-09-10 PM.jpg

Install delivery unit HANA_IM_DP (Data Provisioning)

The specific delivery unit needs to be downloading and upload from the studio or the web interface, this will provide you:

  • The monitoring functionality
  • The proxy application to provide a way to communicate with the DPA (cloud scenario)
  • The admin application for DPA configuration (cloud scenario)

7-11-2015 8-31-14 PM.jpg

7-11-2015 8-31-32 PM.jpg

Upload from the studio

7-11-2015 8-32-45 PM.jpg

Once done assign the monitoring role and add the view from the cockpit

7-11-2015 8-36-00 PM.jpg

7-11-2015 8-48-03 PM.jpg

Install and register Data Provisioning Agent

The Data Provisioning Agent is used to make the bridge between Hana and source system where the driver can’t be run from Hana (DPS) over a pre-build adapter, in some case it allow Hana to write back data into source system.
Use the DPA allow live replication.

The agent is part of the package download earlier

7-11-2015 9-04-37 PM.jpg

Run and installed it as needed

7-8-2015 9-40-08 PM.jpg

Once installed open the cockpit agent

7-11-2015 9-12-27 PM.jpg

Make sure the agent is started, connect and register it to Hana with the necessary adapter

7-12-2015 4-46-46 PM.jpg

Let create the source system in Hana now.

Create remote source

Now that my IQ db is in place and my Hana adapter is installed I will create my source system in SDA where I need to get the data from.

Let start with my IQ database, before create the connection in SDA install and set the lib on Hana server. To create my connection I will use the following statement:

create remote source I841035 adapter iqodbc configuration ‘;ServerName=HANAIQ03;CommLinks=tcpip(host=usphlvm1789:1113)’ with CREDENTIAL TYPE ‘PASSWORD’ USING ‘user=I841035;password=xxxxxxx’;

Once done refresh the provisioning

7-11-2015 10-42-09 PM.jpg

And create the ERP on Hana schema source system by selecting the adapter added earlier

7-12-2015 4-42-33 PM.jpg

7-12-2015 4-59-13 PM.jpg

And check the remote subscription form the cockpit

7-12-2015 5-47-27 PM.jpg

Data replication and monitoring

My remote source connect I will now define which table I want to replicate and how I want it to look like once loaded.

Make sure your user schema is part of _SYS_REPO with “CREATE ANY” granted.

From the development workbench go to “Editor” and select your package and create a new replication task

7-12-2015 6-16-35 PM.jpg

7-12-2015 6-18-23 PM.jpg

And fill the necessary information, target schema, virtual table schema, table prefix and so on.

From detail perspective several option are possible

Add/remove/edit table

7-12-2015 7-08-12 PM.jpg

Set filter

Define the load behavior in order to have a certain level of detail on the change that encore on the table.

7-12-2015 6-56-01 PM.jpg

Partition data for better performance

7-12-2015 7-09-15 PM.jpg

Once you preference are set, save the configuration to activate it.

7-12-2015 7-14-02 PM.jpg

From the monitoring side check the task log

7-12-2015 7-23-36 PM.jpg

Once activate go on the catalog view and check if the procedure is crated as well as the virtual tables/views and table, and invoke the procedure to start the replication

7-12-2015 7-27-51 PM.jpg

I did repeat the same procedure for my ERP on Hana schema, once the procedure is invoked on the remote Hana db we can see additional table created and trigger for the relevant table replicated

7-15-2015 1-38-18 PM.jpg

From a monitoring side, I did add 4 additional user and we can see the apply count

7-15-2015 2-42-17 PM.jpg

The replication is now operational, in my next document I’ll explain how to configure several datasource and construct one realtime report with the input of different table.


You must be Logged on to comment or reply to a post.
  • Hi Williams Ruter


         While importing the HANA_IM_DP Delivery Unit, I am getting the following error

    "A SQL error occurred while creating a SQL view"     .

    Kindly let me know the solution.


    Thanks and Regards,

    Ramya G

    • Hello Ramya,


      Which version of Hana are you on ?

      If you are on SP10 use the DP from the "install and upgrade path"

      If you are on SP9 use the DP SP00 from "support pack and path"



      • Hi Williams,

             I was successful in importing the DP SP00, but when i try to register my Adapter to the agent i get the following error.


        Thanks and Regards,

        Ramya G

        • Hello Ramya,

          Make sure hana and the dp agent server talk to each other with hostname not ip.

          Please le me know if you still have issue.


  • Hi Williams


    Whilst importing the HANA_IM_DP Delivery Unit, I'm getting the following error "A SQL error occurred while creating a SQl view".

    The import of DU HANA_IM_ESS is also ending with errros.

    The HANA version is SP9 (1.00.097).


    What do you suggest in this case?


    Thank you

  • Hi Williams,


    Thanks for this blog.


    I have done all the required things to use SDI with SQL Server remote source.


    I was successful to 'connect to HANA' from dp agent without using 'Hana on Cloud' option but using the 'db tunnel' or cloud connector. Or in other words, using the TCP protocol and creating the inbound firewall rule for the port 5050.


    But actually my HANA is on cloud (HCP). Also it is not always possible to open port in customer db server


    Hence I followed your blog and downloaded & imported HANA_IM_DP (To be exact, as I am on SPS09).


    Now I need to connect to HANA by using 'HANA on cloud' so that no port needs to be open.


    Could you please let me know below 2 things related to 'Connect to HANA' in dp agent,


    1)   Where to get proxy details like (Port name, Port, User & Password)


    I can access '' but it gives no required info. It gives the below;



    • api: "",
    • message: "dpproxy",
    • responseCode: 0



    2)   Is HANA hostname to be provided with account-name ( where acb12 is account-name) or without it


    Connect to HANA using HTTP proxy.png





    • Hello Ramees,

      For the first point, the proxy information is mandatory only if you need to go through a proxy to access your system over the web, in this case this information needs to be provide by your network team overwise uncheck the box.

      For the second point, you just provide your hana hostname, user name, port and password. Do not use ""


      Feel free to let me know if you have any issue or question



      • Thanks Williams for the details.


        I tried as below but ended up in error ,


        Connect To HANA using Hana On Cloud.png

        In webdispatcher.ini, 'icm/server_port_0' is set as 'PROT=HTTP,PORT=8000,PROCTIMEOUT=600'

          • Thanks Williams for the reply.


            Got the below response for telnet 8000


            "Connecting To not open connection to the host, on port 8000: Connect failed."


   being the landscape host, I am not sure it will ever allow this connection. Are you sure we can connect with just HANA Host name as '' and without proxy details for 'HANA on cloud'? Then what is the need of importing the HANA_IM_DP and that proxy service ('')?


            My webdispatcher.ini is attached below. Also I am not very sure what port to be used in this case for the 'Connect to HANA' in dpagent.



          • Hello Ramees,


            First change you webdispatcher configuration, because you will point nowhere if the value is "0", reverse it to the default port or provide a custom open port.


            If you reverse to the default port "8000" please try to connect on your hana instance over http by usining it., if it works then the proxy parameter is not needed.


            The DU HANA_IM_DP provide roles in order to allow you to make the setup over http protocol.


            Let me know the result.



          • I have successfully connected over https following the document provided by SAP support.


            Point to be noted is that, I didn't change anything on the webdispatcher.ini. It got connected on port 433 also you need to provide <account>.<hcp host> (say as host name on dpagent config tool. There were no proxies required in my case.


            As the document was quite long, I am not sharing it here. As per the SAP support, they are in process of updating documentation for dpagent connection over https.

          • Hi Ramees,


            I am facing the same issue. I want to connect the dp agent to HANA on cloud but it says 'Connection to <account> refused'.Can you share the document from which you were able to resolve the issue?

          • Hello Walled,

            If you want to connect you DP agent to Hana on Coud you will have to first get the signed certificate from the Hana and load into your DP server by keytool command.

            1) From the Hana Webdispatcher interface copy the text content your SAPSSLS.pse

            2) On your DP agent server create a SAPSSL.cer file

            3) Copy the content text into the SAPSSL.cer

            4) Use keytool command to add it into the cacerts

            5) On the DP agent config specify the Keystore password and check "Is SSL enable on Hana server"











          • Hello Waleed,

            You don't need to go over the webdispatcher is you already have access to your Hana instance by https.

            So if this is the case, you just need to import your certificate into the DP Agent server, can please tell me what error message you have ?



          • /
            HANA_dp - Copy.PNG
          • Hey Waleed,


            Can you confirm that you can first connect on the Hana cockpit over 443 ? if yes please confirm that you import your certificate with the following command :


            keytool -importcert -keystore cacerts -storepass <password> -file SAPSSL.cer -noprompt


            One important point is format used to import it.


            Please let me know if you have any issue



  • Hi Williams,


    Thanks this is a great blog!

    We are on SPS9 and are planning to install EIM.

    In your blog you mention installing HANA_IM_ESS.

    In one of your replies you state that you only need HANA_IM_ESS for ADP.


    In addition, I have looked at the EIM administration guides for SPS9 and SPS10.

    In the SPS9 guide ESS isn't mentioned.

    In the SPS10 guide ESS is mentioned, but explicitly only for SPS9.


    Now I'm confused. Can you clarify this point for me?



    • Hello Martin,

      ESS provide a semantic search feature required for SAP Agile Data Preparation which allows business analysts to search for data sources by name or content, so if you just want to replicate data and transform it, it's not mandatory.



      • Thanks for your explanation, Williams.


        I still have another question. When I looked at Support Packages and Patches | SAP Support Portal for, I couldn't find it.

        I searched under SAP HANA SDI 1.0. But there I could only find HANAIMDP and the DP agent. I searched under HANA Platform. Didn't find it there either.

        I also tried "HANAIM*" as a search term. But I only got HANAIMDP00 and HANAIMDP01.


        Could you describe the correct path or search term?


        Thanks in advance


  • Hi Williams,


    Thanks for this blog.

    I am facing difficulty in registering an adapter with my HANA server:



    Please help how to proceed.


    Best Regards,


    • Hello Himanshu,

      Which version of Hana are you on ? also please make sure to have the necessary credential to register your adapter in Hana.



      • Hi Williams,


        My system version is

        I think I have all necessary credentials. I at least checked system privilege ADAPTER ADMIN for my user.


        Please suggest what could be the issue.





        Update on 5th Jan 2016 >> The issue does not occur if I specify IP address in agent hostname while registering the agent.

        • Hi Williams,


          The next hurdle that I now face is while creating a remote source using file adapter. I have created a remote source using file adapter but do not see the data files as remote source tables under provisioning remote source in HANA Studio.



          I have my data & CFG files in correct directories specified while configuring the DP agent. My DP agent is installed in the same windows box. But, I am not sure if index server must be specified as source location for my remote source; I do not get another option to choose (agent?).

          Am I missing any configuration step?

          Please help.




          • Hello Himanshu,

            You should be able to read it, the adapter name should appear and not indexserver.

            Can you please have a look at the log file and/or send it to me please ?

            1-5-2016 9-25-50 PM.png

            1-5-2016 9-25-50 PM.png
    • Hello Martin,

      The DP Agent (IMDB_DPAGENT100_02) and IM_DP (HANAIMDP02) are actually made for Hana SP11, however this version is also backward compatible with Hana SP10.

      For more clarity Martin you can also refer to SDI PAM

      Let me know if you have further question.



  • /
  • I am getting quite a few activation errors when installing the ESS DU.

    Example error message:

    A transaction rollback may have left the object in an inconsistent state.


    We will be evaluating ESS with ADP.  Any guidance on correcting this would be appreciated.




      • Hello Williams,


        As it turns out I was deploying the incorrect version for our HANA SPS level (102.3) we are on.  The Admin Guide is not clear on this. I discovered it via the PAM.


        I did an uninstall of those components which did get installed and then installed the correct DU version (HANAIMESS01P_2-70000759) and it was successful.  Thanks for your follow-up.



  • /
      • /
        • Hello Waleed,

          Please check the dpagent_service_evenlog and check the connectivity between the agent and Hana, look if you have the following error :


          # Receive Ping reply message from DP Agent.

          # No agent existing!!!




          # Error #10061 in  connecting socket.

          # Error #10057 in  ping agent.

          # Error #10057 in  receiving from agent.

          # No agent existing!!!


          In some case you just need to restart the dp agent.


          Let me know



          • Hi Williams,


            I checked the dpagent_service_evenlog  file and I see the errors multiple times:


            # Error #10061 in  connecting socket.

            # Error #10057 in  ping agent.

            # Error #10057 in  receiving from agent.

            # No agent existing!!!


            I tried to restart the agent but the error is still the same.

          • Hi Williams,


            Thanks for the information. It worked but the next issue I am facing is that I cannot create a remote source using TwitterAdapter. I am getting the following error:



          • Hey Waleed,

            The source location of your adapter must be the name of the adapter you created for the Twitter.

            From your screen i can see "index server" which is wrong.

            Can you check if you adapter shows up in the drop down list



          • Hi Williams,


            Unfortunately I can only see 'index server' in the Source Location dropdown list. What could be the possible issue?




  • Hi Williams,


    Thanks for your document with detail explanation.


    While I am trying to importing the delivery unit getting the below error.


    Please suggest me to resolve this issue.





    Thanks & Regards,


    • Hello Ramana,

      According your screen picture you seems to deploy DP and ESS version for SP9, can you please confirm which version of Hana you working on ?



      • /
        • Hey Ramana,

          Uninstall both previous DU left off, them re-install the DP first by selecting "Overwrite inactive version" and "Activate objects"



          • Hi Williams,

            I am trying to setup SDI , i have installed the DP Agent on to my laptop and was able to connect to my HANA server. But when i tried do Agent Registration i am getting the error below.




            when i click on the register agent and give a name to my agent i am getting the error below. please advice on what needs to be done.


          • Hello Saikrishna,

            Which OS your laptop is running on ? because the dp agent must run on server (SLES 11 or RHEL)

            Please send me the dp_agent log and framework too.





  • Hi William,


    I recently tried to install the DP Agent version but I get the following error when I install the software:



  • Hello Williams,


    I have recently installed HANA SPS12. I installed the DP Agent on a Windows server - DP AGENT SP03. Everything, including registering the Twitter adapter went just fine.

    But when I tried to create a remote source, after entering my consumer token and access token keys, I got this message -


    SAP DBTech JDBC: [403]: internal error: Cannot get remote souce objects: connect timed out.


    Can you help me?



    • Hello Martin,

      Can you check if the firewall of your windows server is stop ? also please can you send me the dp_agent log and framework log too ?




  • Hi Williams,


    Great Blog you have here!


    For example, assuming that this tool will be used moving forward to all HANA/S4 implementation and assuming the existing client is currently using BODS and would like to migrate to this tool (SDI)), what are some of the factors/things that we can consider in migrating objects/flow. Are there any tool(s) for migration from BODS to SDI or it would be an effort like creating new flow to SDI base from BODS?




    • Hey R-jay,

      This is a great question, in term of effort (flow based), if a customer wants to move from DS to SDI, since no tool (as far as know) exist to migrate the current flow build in DS; the logic will need to be rebuild in Hana directly.

      Since there is no portability of the flow, the customer will have to evaluate the way the logic is built in DS vs the way he wants to do it in Hana.

      Let me know if you have other question



      • Thanks for that informative response.


        Are there or do you have any guide questions for this type of approach considering that there's no available tool to migrate it.


        With this initially, i am considering that a flow consist a source(s), the logic (dev work in SDI) and the target. and I also defined it into 3 areas of complexity like simple, medium and high. Do you have like high level definition how these complexities are defined and more likely how long to develop a simple, a medium and a high base from your expert advice.





  • /
    • Hello Hari,

      from your agent server, are able to resolve the name of you hana server and vise-versa ?

      Please run some nslookup command, if it failed them you will need to add entries in hosts file of both server



      • Hi Williams,

        I have DP agent installed on my windows 10 laptop and i could able to connect to HANA server . but getting the same error as above when i try to register the Agent. tried both from DP Agent and HANA server using SQL command. i could able to ping the HANA server from my laptop and came to know that Server needs to be able to communicate with the agent (laptop) on port 5050. So the security guys opened  it on the server one way (server to dp agent).  anything else i am missing.?? does port 5051 needs to be opened as well.? bith 5050 and 5051 are listed on the laptopn.



  • Hi William,

    I need to understand, can SDI/SDQ can be used for address standardization and duplicate check in MDG?

    Do we have a use case of SDI/SDQ supporting MDG?

    MDG instance is running on HANA.

    Please suggest.

    Thanks and regards
    Abhimanyu Singh

  • Hi Williams,

    Thank so much for this blog  and  for keeping posting comments.
    My question is about : OracleEccAdapter Credentials.

    Do i need a special user from the database when using the OracleECCAdapter
    or a Sapgui’s User would suffice as it is for the AbapAdapter ?

    Thanks in advance,


    • Hello Esteban,
      You are very welcome, yes you will need a specific user in Oracle because you will need to run a script where a dedicated oracle user needs to be defined.
      Read my other document where i explain it :

      Let me know if you have any question


  • Hi,


    good blog! Can I install more than one DP agent on a windows machine? Right now, I am running into issues that I have to uninstall the existing oneto have another one installed. I would like to use this system to serve more than one agents serving more than one HANA.

    • Hello Venu,

      On Linux, you can install multiple agents on a single host machine but on Windows, only a single

      DP Agent installation is supported per host.



  • Hi Experts,
    I am facing issues while registering the HANA Adapter, As you can see in below screen shot i can connect to HANA Server using port 30015 and Agent registration is also successful using IP address.
    Inline image 1
    framework.trc logs are as below..
    2018-01-06 18:35:22,218 [ERROR] DPFramework | WorkerThread.processRequest  - Executing SQL statement "create adapter "HanaAdapter" at location agent "Agentdp"" failed with SAP DBTech JDBC: [403]: internal error: Cannot get adapter capabilities: exception 151044: Agent "Agentdp" is not available.
                                     Context: SAP DBTech JDBC: [403]: internal error: Cannot get adapter capabilities: exception 151044: Agent "Agentdp" is not available.
    dpserver logs showing as below:
    Generic stream error: getsockopt, Event=EPOLLERR - , rc=104: Connection reset by peer; $channel$={<NetworkChannel>={<NetworkChannelBase>={this=139883525306904, fd=89, refCnt=1, idx=0, local=, remote=, state=Connected, pending=[r---]}}}
    exception throw location:
     1: 0x00007f39d95685d1 in Stream::NetworkChannelCompletionThread::run(void*&)+0x2b0 at NetworkChannelCompletion.cpp:481 (
     2: 0x00007f39d949ad56 in Execution::Thread::staticMainImp(void**)+0x412 at Thread.cpp:537 (
     3: 0x00007f39d949c908 in Execution::Thread::staticMain(void*)+0x34 at ThreadMain.cpp:26 (
    [96838]{-1}[-1/-1] 2018-01-06 13:43:52.550653 i TraceContext     TraceContext.cpp(01017) : UserName=, StatementExecutionID=21474836481
    [96838]{-1}[-1/-1] 2018-01-06 13:43:52.550650 e DPFramework      AgentPool.cpp(00922) : exception 151044: Agent "Agentdp" is not available.
    [96795]{-1}[-1/-1] 2018-01-06 13:45:36.465716 i Savepoint        SavepointImpl.cpp(02778) : Savepoint current savepoint version: 693, restart redo log position: 0x124443, next savepoint version: 694, last snapshot SP version: 0


    Inline image 1