Integrating SAP HANA with Hadoop – all you always wanted to know

After publishing a post called 3 tips to survive as a SAP BI consultant in 2016 and another one called Enterprise Data Warehouse + SAP Vora = High Value, I got a great question from Sivaramakrishnan M basically asking for some documents or links that could help him get started with SAP HANA and SAP Vora integration. After a quick search I realized it was not really easy to get hold of all information you need, so here it is: all you always wanted to know about SAP HANA and Hadoop, but was afraid to ask.

Before we get started on the link material, we can benefit from some context. I’m assuming you already know what SAP HANA is. SAP In-Memory plataform that allow real time transactional and analytical processing, composed of several different enginnes (Geospacial, Predictive, Business Function Library and so on) which allowed for a huge transformation on what we are able to do. More information can be found here at SCN: SAP HANA and In-Memory Computing

According to Hortonworks, the definition of Hadoop is: “… an open source framework for distributed storage and processing of large sets of data on commodity hardware. Hadoop enables businesses to quickly gain insight from massive amounts of structured and unstructured data”. And for those really interested on how Hadoop (Hortonworks) architecture looks like, I’d suggest the following link: http://hortonworks.com/hadoop/ and there you will find the following image:Having the concepts of each component clear, next step is to define what the integration between those two components could look like. It basically will depend on the use case you have:

• Smart Data Access –> in case you need to read data out of Hadoop, you can use SAP HANA Smart Data Access (SDA) to do it. SDA is widely used when it comes to hybrid models (SAP HANA + SAP NetWeaver BW powered by SAP HANA) or even Near Line Storage (NLS) scenarios. You can basically access a “table” in a different repository (mainstream databases all included) from SAP HANA without actually having to bring the data over to SAP HANA. So you could have your “hot” data in SAP HANA and your cold data into Hadoop and using SDA a simple UNION would bring data from both “tables” together.
• SAP BusinessObjects Universe –> in case you only need to report in Hadoop data out of SAP BusinessObjects Suite, you can combine data from any source to Hadoop using the Universe, SAP BusinessObjects semantic layer to get the job done. There you can setup relationships, rules, etc.
• SAP DataServices 4.1 (and above) –> in case you really need to bring data from Hadoop into SAP HANA and maybe apply some heavy transformation on the way, that is your path to go. SAP DataServices have been tunned to been able to read and write huge amount of data both ways.
• SAP Lumira –> in case you only need front-end intregration and less complex data handling and transformation, that is a easy way to go. SAP Lumira can access and combine data from Hadoop (HDFS Data Set, Hive or Impala Data Set or a SAP Vora Data Set) and SAP HANA.
• SAP Vora –> in case you need to correlate Hadoop and SAP HANA data for instant insight that drives contextually-aware decisions that can be processes either on Hadoop or in SAP HANA

With all those use cases in mind, Hortonworks draw a great picture of how the architecture could look like:

With all that clear, I believe we can jump directly to the main topic of this post. Please, find below usefull links and its descriptions to bring you up to speed when integrating SAP HANA with Hadoop.

I’m very confident that once you reach the bottom of this post (and visit all or at least most of the links I have compiled here) you will be able to get your SAP HANA and Hadoop integration going. In case you have additional links that I should include on this post, please, let me know via comments and I’ll be more than glad to add them here.

All the best,