HANA datasources – Real-Time Replication into BW using SAP HANA EIM Smart Data Integration
As you may know , from SP09 of HANA we have the EIM services now being made available as part of HANA as SDI and SDQ . These services allow you to load data from external DBs either virtually through the federation feature which we call as SDA (Smart data access) or replicate in realtime through the SDI (Smart data Integration) functionality.
More Info on HANA EIM: HANA IM Services: SDI and SDQ in HANA SPS09
Especially interesting for BW is the HANA Smart data Integration .To use the SDI functionality, the external DBs from which data needs to be replicated are connected through the SDI adapters in the remote source which acts as a channel for realtime replication.SDI supports all types of data delivery reatime, batch and federation.
Platform Integration – HANA Smart Data Integration for SAP BW
To connect to the EIM services of HANA to BW we have a new source system type called as SAP HANA source systems. This new source system replaces the existing DB connect source systems from 750 codeline . It has the following capabilities
- it leverages SAP HANA Smart Data Integration as provisioning technology
- it provides a simplified and unified source system connection to SDI, SDA and local SAP HANA sources with the HANA Source System in BW
- it uses the SDI Adapter in SAP BW for real-time replication from various source DBs
There are many databases that can be connected through SDI or SDA . Only a few are listed in the above fig. The complete and the up-to-date list can be accessed here : full list of supported adapters
SAP HANA source systems comes in 3 flavors.
HANA_LOCAL: this type of source system allows connection to schemas of the same database.
- is a type of source system connection that allows connection to external DB and access the data using the virtual table through the SDA functionality and which allows realtime replication through the SDI adapter based remote source connection
- for Multi-tenant Database Container (MDC) support
- Support for connecting to SAP HANA tables / views in other database tenants
To create a new source system for SDI based replication, first a remote source connection to the external database using the SDI adapter has to be created. All SDI adapters are hosted on a DP agent : Install and configure a DP Agent. Then the remote source is pointed to the source system while creating a new one (see below):
So, once the source system is created it will be possible to create HANA datasources for these source system. Each HANA datasource of SDI type during activation creates a target table for data replication along with other HANA artifacts in the BW schema.
Leveraging the SAP HANA SDI HANA Adapter in SAP BW-Integration Scenarios with SAP BW
The HANA datasource can be enabled for 2 types of replication: UPSERT and INSERT.
UPSERT replication : During replication the target table of the datasource holds the actual of the data as in the source table , no history is captured. only updated information is stored.
INSERT replication: if the datasource is enable for INSERT replication every update on the source data is captured along with its history which means the before and after image as also stored in the target table which is captured by the SDI adapter which is later mapped to the 0RECORDMODE IOBJ in the transformation during data modeling.
This activated datasources can be connected to the Open ODS view for ad-hoc and quick reporting or to an ADSO for traditional data staging scenarios of BW.
Real-time replication up to the data staging layer of ADSO can be enabled with a new feature in Process chains Called as Streaming process chains. More details are in the video links below.
Enable realtime replication of type UPSERT and consume in an open ODS view
Enable realtime replication of type INSERT from datasource to ADSO using Streaming process chains
How to set up HANA datasource for SDI
Great post, thank you, Mary!
Provides a nice clarification for the whole EIM SDA.
Could you please tell whether it would be somehow possible to create SDA DataSources based on the standard (for example LO – LIS) extractors streaming the data to (V3) and from RSA7 queue via INSERT replication? Kind of RDA daemon replacement. Or is it only for the tables and views?
The recommended approach for real-time streaming from SAP sources would be to go for ODP datasources and including their dataflow in the streaming process chain for real-time replication. SDI would not be necessary in this case.
Thanks so much for you detail explanation.
I was wondering in case you have to chose in a new BW Hana implementation the best method to replicate data into BW systems based on a real-time need.... What option would be the best and why?
Can you please elaborate the approach you have suggested above. Do you mean to say all the ODP datasources can send data at run time directly to SAP BW unlike many standard datasources?
SDI is used mainly for replicating data in realtime from non-SAP sources.
For standard extractor based approach we still use ODP type of source systems for which PSA is not longer mandatory.
Thus, you can load directly to the BW providers.
Streaming process chains enabled for BW extractors (ODP) run only in pull mode. Meaning the job can run at highest frequency of 1 minute.
Whereas with SDI adaptar and SLT-ODP, the process chains run in push mode.
The advantage with push mode is that, it won't create unnecessary request (TSN) in BW targets. Is it possible to overcome this problem with BW extractor-ODP?
Thanks for sharing your experience.
What will be your recommendation on extracting data out of SAP either BW or ERP where both the systems are running on SAP HANA. Need of customer is to extract data and use it in 3rd party external system for analytics in near real time.
Usually Open Hub destinations are used to extract out of SAP BW systems and feed it to third party tools.
Thanks and Regards,
Thanks for the article.
I'm wondering whether we can connect SAP HANA(as source) to other database using SDI. Do you have any idea about this?
SDI is used to connect to HANA as target DB from many external DBs.
Is connection to any database via SDI adapter covered under BW license? If yes, does it automatically mean that EIM in its entierty is available alongwith BW purchase?
As far as I know, purchasing EIM requires a separate license (in addition to HANA platform)
last 2 video links are not working for me.
Please real answare to this question.
What are prerequisites for this requirement?
B. The process chain is running in streaming mode.