Data Coherence Between SAP HANA and Microsoft Azure - Part 1 : Transfer data from Azure Blob to SAP HANA Cloud using SAP Data Intelligence

[feng_liang]

Regardless of where your data is, there is a method to move your data into SAP HANA. Our Customer Innovation team has compiled this blog series to provide step-by-step instructions for moving your data.

In this article, we will talk about solutions we identified for a requirement around transferring incremental data (csv format) from Azure blob storage into SAP HANA Database on cloud using SAP Data Intelligence. When researching solutions within SAP channels, we found that there was no standard process for addressing this, so we wrote this article in hopes that people in a similar situation will find it useful.   

Here are the solutions we tried:   

1. 
   

   RESTful Service
   

   
   We can transfer data between SAP HANA Cloud and Azure with a customized python operator where we write python code to access any service, as long as relevant python sdk libraries are provided. Because this process requires more engineering effort than the following two ways we are going to walk through, it might not be your preferred method.


2. 
   

   Structured Data Operators from SAP Data Intelligence
   

   
   If you have a single csv file that has a stable name that contains the incremental data you need, you may consider Structured File Consumer and Table Producer to read from Azure blob and save into SAP HANA Cloud database table. You can either manually configure your Azure blob connection and SAP HANA database connection inside your operators or preconfigure the two via the SAP Data Intelligence connection manager.


3. 
   

   SAP HANA Client Operator from SAP Data Intelligence
   

   
   If you have a dynamic number of files with changing content, you should consider using monitor file operator and then use SAP HANA Client operator to save the file content into SAP HANA Cloud. One thing that should be mentioned is SAP HANA Client operator currently has a small issue which will be fixed in Q3 or Q4 of 2020.

In conclusion, these are the three ways we found to address the issue of transferring data from Azure Blob to SAP HANA, and each has its strengths and weaknesses depending on the scenario.

Thanks for reading. If you have any questions, please reach out to ci_sce@sap.com.