ETL integration options in SAP BW/4HANA explained

[Jan_van_Ansem]

SAP’s flagship Data Warehousing application BW/4HANA has broken with the tradition of application level integration with external ‘Extract Transform and Load’ (ETL) tools. Integration with tools like SAP Data Services and HANA Smart Data Integration now takes place in the HANA layer, outside the BW/4HANA application. Jan van Ansem explains what the consequences might be if you are planning on using an ETL tool for your BW/4HANA interface.

SAP Data Services is often used by customers to transform and load data from non-SAP systems into the SAP Business Warehouse (BW or BW on HANA, BWoH) application. Data Services allows you to easily connect to a wide variety of data sources and design complex transformations in an easy way, using a graphical user interface for designing ETL processes. I have worked with Data Services for many years and developed a great appreciation for this ETL tool. I was even more happy when SAP introduced Smart Data Integration (SDI), which provides the same functionality as Data Services but is an integrated part of the HANA platform. It came as a bit of a shock to me when I realised that in BW/4HANA, integration with tools like Data Services and SDI has moved outside the BW application layer. ETL integration now takes place in the HANA native layer, which is an architectural improvement, but for existing Data Services users for BW this imposes both license implications and conversion challenges.

The good news: architectural improvements

SAP have delivered on their promise to simplify BW. Integration with ETL tools is easier, more aligned with Enterprise Data Warehouse standards, and far more flexible than it was when integration took place through PSA tables. This is mainly achieved by removing the PSA layer from the BW/4HANA application and using native HANA tables instead.

HANA tables have clear benefits over PSA:

• HANA tables can be used by any application, where PSA tables can only be used by the BW application


• Connectivity to HANA tables is managed through the HANA platform, providing many different mechanisms to load and update tables, whereas PSA tables can only be loaded through the available options in the BW application, of which there are far fewer.


• PSA tables are a BW application specific feature and housekeeping on them is often overlooked. The move to HANA tables means the architecture is more aligned with general data warehouse architectures and management of HANA tables is much easier than managing PSA tables.

The trend to move from Enterprise Data Warehouse (EDW) to Big Data Warehouse (BDW) means that the integration of data from a wide variety of sources becomes more important. The integration of the traditional back-end systems for EDW (S/4HANA, and other SAP source systems) still takes place in BW/4HANA directly through the Operational Data Provisioning (ODP) framework. Most other sources can now easily be integrated in BW/4HANA through the HANA native layer. This is great news for the data warehouse architects and engineers, but possibly not such good news for those who pick up the bill, as I will show in the next paragraph.

The challenge: Licensing

With BW/4HANA, it is no longer possible to use external ETL tools to load directly into the BW/4HANA application layer. Integration with ETL tools takes place in the HANA native layer but creating target tables in the HANA layer directly is not permitted under the HANA runtime license for BW/4HANA. A HANA Enterprise license, which would allow you to create target tables in HANA native, would make your Data Warehouse significantly more expensive. So, what are the options? Let’s have a look at a few different scenarios.

Scenario 1: Big Data Warehouse with Smart Data Integration

If you are planning to grow your EDW (reporting on standard back-office applications) to a BDW (combining back-office data with social media, sensor data and other Big Data sources) then you are looking to combine the capabilities of BW/4HANA with HANA platform capabilities. In this scenario you’ll want a HANA Enterprise license. This allows you to use SDI for your ETL, you can create as many HANA native tables for your BW/4HANA inbound layer as you like and of course you have many other great features of the HANA platform at your disposal.

I expect this will become a more common scenario in the near future, when organisations realise the benefits of the tight integration between traditional EDW and the Big Data Warehouse platform.

Scenario 2: BW/4HANA with SAP Data Services as an ETL tool

Many customers are currently on BW or BW on HANA (BWoH) and use SAP Data Services as an ETL tool. When moving to BW/4HANA, the Data Services processes will need to be converted. Instead of loading to the BW application directly, Data Services will now load into HANA tables. Besides the fact that it will be a considerable effort to change all Data Services processes, there are license implications as well. A runtime license does not allow you to create HANA native tables directly, so customers are faced with the cost for an HANA enterprise license.

SAP have confirmed that they are working on a solution to overcome this licensing issue. More news about this is expected later this year. I can only hope that SAP will not only resolve the licensing issue but also come up with a conversion program to replace the BW targets in dataflows with HANA targets.

My advice for those customers who are now on BW or BWoH and are thinking of moving to BW/4HANA is to discuss their options with their SAP account manager.

Scenario 3: BW/4HANA with other ETL tools

If you are using a non-SAP ETL tool for loading non-SAP data to BW or BWoH and you are thinking of moving to BW/4HANA then you are in the same boat as Data Services users: You can no longer load into BW/4HANA directly, but you will have to load to a HANA native staging layer.  The difference is that it is less likely that SAP will come up with a discounted licensing model to soften the blow. Again, speak to your SAP account manager to discuss your options.

The ETL integration options explained

The question how to use Data Services and SDI with BW/4HANA has frequently popped up on SCN and other discussion forums. The answers have not always been helpful. Different sources sometimes had contradicting information. I hope this blog has made clear that ETL tools no longer integrate with the BW/4HANA application directly. Integration takes place in the HANA native layer. From an architectural perspective this is an improvement, but in some cases, there are license implications. These licensing questions are best discussed with your SAP account manager. For any other questions about ETL and BW/4HANA please feel free to get in touch!