Skip to Content
Author's profile photo Ashish Bahl

Use Case to S/4HANA Smart Data Integration (SDI)

First things first, why there was a need to develop SDI?

Every new technology or platform is developed as a need to address the challenges faced by older technologies, cater to the unmet needs of the consumer or to facilitate a change by making the processes easier and/or economical. Smart Data integration is no exception.

A few years ago, if I wanted to load a flat file into HANA, I had to install Data Services, an ETL tool to replicate, transform and load the data from source to the target system. In order to use the real-time functionalities, I additionally installed SLT/SRS and in case I was not copying data to the source, I used SDA to take advantage of the virtual tables. So, in total I was installing 3 different tools with their own connectors, in-fact I had little choice until Smart Data Integration was developed.

The best of all world, combining all the features of ETL, real-time replication, and virtual tables: Smart data integration acts as a bridge between the source and HANA, reads the source data and translates the values into HANA datatype value. It Batch load and/or replicate changed data in real-time from source to SAP HANA tables providing immediate visibility into conditions located in either SAP HANA or source.

Use Cases:

This use case is focused on how SDI facilitates real-time replication of data from Salesforce to SAP HANA. The data can be either pulled on demand when any query executes or when data in Salesforce changes or gets updated, it gets automatically pushed into SAP HANA.

Advantco International LLC (Advantco) offers SFDC adapter and Microsoft CRM adapter for SAP HANA Smart Data Integration (SDI), its purpose is to batch load or to replicate changed data in real time from Salesforce.com or Microsoft CRM to the SAP HANA tables.

Why SDI is the preferred tool?

  • Easy Configuration & Setup

All that needs to be done is to install the SAP SDI Agent and deploy the SFDC adapter on it. The SDI adapter is hosted on an S/HANA data provisioning agent and acts as a bridge between Salesforce and HANA, reads the source data and translates the values into HANA datatype value.

  • Authentication with the Source system

The SFDC Adapter for SDI supports the basic authentication. All you need to do is provide your credentials (Username, Password, Security Token (Optional)) in a remote source configuration.

  •  Session Reuse Mechanism

The SFDC Adapter for SDI supports Salesforce session reuse in order to reduce the number of login call to Salesforce.com. Salesforce session reuse will automatically renew the session if the current one is expired.

  •  Adjusts Communication Method Dynamically Based on Data Volume

The Adapter provides a “Maximum Expected Number of Record” parameter which is designed to switch automatically between Salesforce APIs. SOAP API is being used when the actual number of records in source data is less than or equal to the value of the maximum expected number of records parameter. Otherwise, BULK API is being used.

  • QueryLocator Results

The QueryLocator is used when querying a large number of records. The QueryLocator represents the server-side cursor that tracks the current processing location in the query result set. The Adapter will call the query repeatedly until all records in the result set have been processed.

  • Virtual Table as a Source

As the SQL query runs directly in the source system, using the SDI Adapter you can virtually access the data without the need to copy the data to S/4HANA.

  • Support batch and real-time for all sources

SDI has the capability to do both near-real-time data replication and an option to execute even in batch mode.

  • Allow transformations on batch and real-time data

SDI as the capability to integrate and cleanse data. The SAP HANA SDI provide SAP HANA Web-based Development Workbench to create flow graphs to transform data. The flow graph enables ETL based batch and Real-time data flows. There are a wide variety of options available to meet the transformation data such as filtering, joining, aggregating, cleansing and more.

  • Change Data Capture (CDC)

Replicating entire source data to the target every day puts an additional load on the non-critical tasks, affecting the performance of the whole system. CDC tracks the change data and transfers the latest changes across source and target, thereby enhancing efficiency and performance.

  • Data Quality

Using SDI, you can be rest assured of both the data quality and data reliability which enhances your experiences in the intended uses in operations, decision making, and planning.

  • Predictive Analytics

Along with integration, SDI can be used to draw meaning insights based on predictive patterns and using machine learning and other algorithms.

  • Provide one connectivity that supports all

SDI provides lots of flexibility to connect to both RDBMS and unstructured data sources. Furthermore, it also supports Big data and other new systems.

  •  Provide one UI that supports all

The SAP HANA SDI provide SAP HANA Web-based Development Workbench which allows you to develop entire applications in a Web browser without having to install any development tools. It provides an intuitive UI and simplifies development by providing many convenient functions.

How it works:

1.How to replicate Account data from Salesforce.com in HANA tables

The SFDC Adapter for SDI acts as a bridge. It opens a connection to the Salesforce and read the source data and translate the values into the Hana datatype value.

Account Retrieval Use Case

The goal is to make this data of Account on Salesforce available in SAP HANA by Querying the remote Account via the Virtual table (data is not physically moved to the cloud, but remains in its original source.)

Create a Remote Source

Save and click Test connection:

We can browse the metadata tables which are provided by the adapter, each table corresponds to one sObject.

2. Grant privileges for _SYS_REPO

Execute the SQL statement below:

GRANT CREATE VIRTUAL TABLE, CREATE REMOTE SUBSCRIPTION, DROP ON REMOTE SOURCE “Salesforce” TO _SYS_REPO;

The result as below:

 

  3. Create a Virtual Table

After creating a Salesforce remote source, we can create a virtual table to retrieve data for an account as below:

Input virtual table name and select “ADVANTCO_SFDC” for schema

Go to “ADVANTCO_SFDC” schema and refresh “Tables” node:

 

4. Query on Virtual table

Open the SQL console, input and run the SQL statement with the result as below:

 

5. Account Replication Use Case

The goal is to make this data of Account on Salesforce available in SAP HANA by using replication task feature

5.1. Create a Replication Task

Create the Replication task as below:

 

Input the name:

Select the remote source, the target schema:

Select “Account” remote table and choose “Initial load only” for the replication behavior.

5.2. Run replication task

After saving the replication task, click run task with the result as below:

5.3. View result

After the replication task done, go to the “ADVANTCO_SFDC” schema to open content of “RT_Account” Table with the result as below:

 

Conclusion:

The SFDC Adapter for SDI and Microsoft Adapter for SDI provides business with the tools to acquire, service, and retain new customers. By delivering a comprehensive data integration and customer data management tool that takes advantage of the best of SAP and SFDC/ Microsoft worlds, businesses can ensure that their Finance, Marketing, Sales, Customer Service, and Fulfillment departments collaborate with each other and get the right information at the right time.

Assigned tags

      16 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Ajay Gupta
      Ajay Gupta

      How the process work for BW on HANA ?

      Ajay

       

      Author's profile photo Ashish Bahl
      Ashish Bahl
      Blog Post Author

      Hi Ajay,

      It works the same for BW on HANA. Let me know if you require additional details or demo.

      You can also connect with me directly at abahl@advantco.com

       

      Ashish

      Author's profile photo Akshay Bhandari
      Akshay Bhandari

      Hello,

      Can we use SDI for replication of data from S/4 HANA to Tera data?

      Regards

      Akshay

      Author's profile photo Manjari Shrivastava
      Manjari Shrivastava

      Hi,

      Can we use it to connect and replicate Excel data real time to SAP HANA?

      Author's profile photo Suresh Kancharla
      Suresh Kancharla

      I also have the same question. Can please someone educate me if this can be used to load DATA to HANA server from Excel or other sources. I am looking for ETL capabilities and want to check if data from Source system can be cleansed, transformed and loaded to S4 HANA using BAPI or IDOC(ex: BP as Customer or Vendor) . I know Data services can take care of it. But we are also checking for SDI capabilities.  Your response will be much appreciated.

      Author's profile photo Matthias Maier
      Matthias Maier

      I have recently seen an Excel Adapter. I guess it is for the purpose of loading Excel Files via SDI into HANA. However, I have to say, I haven't tested so far.

      HANA%20SDI%20Adapter

      HANA SDI Adapter

      Author's profile photo Viral Naik
      Viral Naik

      Can we connect to sales force using SDA? Is SFDC adapter available in SDA?

      Author's profile photo Ashish Bahl
      Ashish Bahl
      Blog Post Author

      Hi Viral,

      Yes, you can connect to Salesforce using Salesforce Adapter for SAP HANA Smart Data Integration.

      Below is the SAP Store listing:

      https://store.sap.com/en/product/display-0000027111_live_v1/Salesforce%20Adapter%20for%20SAP%20HANA%20Smart%20Data%20Integration

      Please let me know if you have additional questions or require a demo to discuss your use case.

       

      Best,

      Ashish

       

      Author's profile photo Suresh Kancharla
      Suresh Kancharla

      Can please someone educate me if this can be used to load DATA to HANA server from Excel or other sources. I am looking for ETL capabilities and want to check if data from Source system can be cleansed, transformed and loaded to S4 HANA using BAPI or IDOC(ex: BP as Customer or Vendor) . I know Data services can take care of it. But we are also checking for SDI capabilities.  Your response will be much appreciated.

      Author's profile photo Rao Bheemarasetty
      Rao Bheemarasetty

      Hello Suresh,

      I'm not an expert in this subject but based on this blog and your follow-up questions I came across the following document link that provides solutions you are looking for

      https://help.sap.com/viewer/71c4a6e6b4dc4a5ab3e17bb1d7e98104/2.0_SPS04/en-US/55920ea404c048a3ab4ba1501a66713d.html

      B R

      Rao B.

      Author's profile photo Ashish Verma
      Ashish Verma

      Hi Ashish, Is it feasible to use SDI to replicate data from S/4 HANA to  Hadoop or Teradata.

      Author's profile photo Werner Dähn
      Werner Dähn

      Technically yes. You create a virtual table for your target system and then execute an "insert into <virtual_table> select * from <hana_table>".

      If you mean with replicate a realtime replication, then you must treat the Hana database as another external source as only a virtual table supports realtime subscriptions. So it is a replication from virtual table to virtual table hence.

      But I expect you will not be happy with the solution. SDI is optimized for getting data into Hana, not into external systems. Better build a solution manually.

      Author's profile photo Ashish Verma
      Ashish Verma

      Thank you.  Any recommendations/suggestions for data replication from S/4 to Teradata or Hadoop.

      Author's profile photo Werner Dähn
      Werner Dähn

      Depends on what your requirements for replication are and the processing engines you have available.

      So what is your ultimate goal? Have the S/4Hana data in the other system as well with a latency of sub seconds? Dump the S/4Hana table once a day? Why even move the S/4Hana data out to something else?

      These insights will help to suggest options.

      Author's profile photo Ashish Verma
      Ashish Verma

      Requirement is to get data from all sources (including S/4 HANA) to  Hadoop / Teradata in as realtime as possible. Any further transformation will be done on target platform. You are right, there are plenty of examples examples from any source to S/4 HANA however wasnt able to find an example where Data was replicated from S/4 hana to other system using SDI.   Thanks once again

      Author's profile photo Werner Dähn
      Werner Dähn

      The classic option to get data out of SAP is SAP SLT but that supports neither Teradata nor Hadoop as target.

      Hadoop has a secondary challenge and that is the number of files - you do not want to create a new file for every transaction in the source database but collect multiple. So that needs deltalake or a similar approach.

       

      What I prefer (and provide solutions for) is S/4Hana to Kafka and from there everybody, including Hadoop, can consume the data at their will.