Hello All,
I am writing this blog post on HANA Smart data integration aka SAP HANA Flowgraphs. Basically SAP HANA Flowgraphs are nothing but ETL data flows which are inbuilt in SAP's web based modelling toll i.e. SAP WEB IDE or SAP Business Application Studio.
A flow graph is usually considered an operator which helps transform data from a remote source into SAP HANA either in batch or real-time mode.A flow graph performs a configurable series of data transformation operations, such as joining, filtering, cleansing, masking, aggregating, and so on.HANA flowgraphs are design time artifacts and can be called inside as stored procedure which I will mention in this blog along with flowgraph debug, performance check etc,
But, a flow graph can also be used to input data from a calculation model to a template table. It can also be easily scheduled. By doing this, the calculation model output can be persisted without writing any code and the inbuilt HANA functions can take care of the rest.
Attached below are the available Transformations in SAP SDI HANA FLOWGRAPHS.
Here I am not going to discuss about what are the prerequisites required to set up SAP SDI to load into SAP HANA either in batch or real-time mode. I will discuss how to debug and what happens in debug.
But before to that I want to explain how a folder structure in SAP WEB IDE and SAP BAS looks like when we create design time artifacts such as Flowgraphs, Stored Procedures, Table Types, CVs, Table Functions etc. in HANA which can later be consumed as runtime artifacts.
When we create a HANA flowgraph it is created with extension as .hdbflowgraph. Once the flowgraphs is created you can see them in TASKs by connecting to the HDI container.
SAP web ide WORKSPACE folder structure : -
Open the flowgraphs form the folder it seems as below. I have displayed a demo flowgraph here.
Now go to the properties button apply the settings and Variables. Those who come from SAP BODS background for sure they will know about JOB variables etc.
Now on the debug mode and do the data preview on all the nodes. Please follow the below image shared with instructions
Once the flowgraph is saved we need to build the flowgraph. To do that we can go to the DB folder on the work space and click on Build as shown in below picture.
Once the build is successful we can do that data preview as respective transformations of the flowgraph.
what happens when we click on data preview. When we run the flowgraph on debug mode and do the data preview there are Calculation Views are created and these calculations views are stored in HANA DB which need to be cleaned. I have written this in one of my other blog for HANA invalidated views. Please find the complete information of the blog here
https://blogs.sap.com/2022/03/03/sap-hana-xsa-find-invalidated-viewsfunctionsprocedures-and-handy-qu...
For example let us say I clicked on Data View of the History preservations Transform. What is happening in the background please see in below picture.
when we click on the SQL we can see that a select statement is generate with select all the columns FROM as the _CV.
I cannot post the SQL as it is my code but you can test the same.
suppose 2 records are not loaded to the target. Then in that case we can compare the CV generated by flowgraph with the target table and see what is missing.
Once the flowgraph is created we can call the fowgraph inside a store procedure call with START TASK by passing the parameters.
Also we can see the flowgraph executions on M_TASKS table inside HANA DB and also can see where in the transformation HANA flowgraphs is consuming what time and what kind of SQL is generated at each level.
you can refer to my blog post
https://blogs.sap.com/2022/03/03/sap-hana-xsa-find-invalidated-viewsfunctionsprocedures-and-handy-qu...
is n't it powerful guys ?
Please test it and let me know your views in comments section at the end of the blog.
Thank You
Narasingha
I am writing this blog post on HANA Smart data integration aka SAP HANA Flowgraphs. Basically SAP HANA Flowgraphs are nothing but ETL data flows which are inbuilt in SAP's web based modelling toll i.e. SAP WEB IDE or SAP Business Application Studio.
Flow Graphs
A flow graph is usually considered an operator which helps transform data from a remote source into SAP HANA either in batch or real-time mode.A flow graph performs a configurable series of data transformation operations, such as joining, filtering, cleansing, masking, aggregating, and so on.HANA flowgraphs are design time artifacts and can be called inside as stored procedure which I will mention in this blog along with flowgraph debug, performance check etc,
But, a flow graph can also be used to input data from a calculation model to a template table. It can also be easily scheduled. By doing this, the calculation model output can be persisted without writing any code and the inbuilt HANA functions can take care of the rest.
Attached below are the available Transformations in SAP SDI HANA FLOWGRAPHS.
Here I am not going to discuss about what are the prerequisites required to set up SAP SDI to load into SAP HANA either in batch or real-time mode. I will discuss how to debug and what happens in debug.
But before to that I want to explain how a folder structure in SAP WEB IDE and SAP BAS looks like when we create design time artifacts such as Flowgraphs, Stored Procedures, Table Types, CVs, Table Functions etc. in HANA which can later be consumed as runtime artifacts.
When we create a HANA flowgraph it is created with extension as .hdbflowgraph. Once the flowgraphs is created you can see them in TASKs by connecting to the HDI container.
SAP web ide WORKSPACE folder structure : -
Open the flowgraphs form the folder it seems as below. I have displayed a demo flowgraph here.
Now go to the properties button apply the settings and Variables. Those who come from SAP BODS background for sure they will know about JOB variables etc.
Now on the debug mode and do the data preview on all the nodes. Please follow the below image shared with instructions
Once the flowgraph is saved we need to build the flowgraph. To do that we can go to the DB folder on the work space and click on Build as shown in below picture.
Once the build is successful we can do that data preview as respective transformations of the flowgraph.
what happens when we click on data preview. When we run the flowgraph on debug mode and do the data preview there are Calculation Views are created and these calculations views are stored in HANA DB which need to be cleaned. I have written this in one of my other blog for HANA invalidated views. Please find the complete information of the blog here
https://blogs.sap.com/2022/03/03/sap-hana-xsa-find-invalidated-viewsfunctionsprocedures-and-handy-qu...
For example let us say I clicked on Data View of the History preservations Transform. What is happening in the background please see in below picture.
when we click on the SQL we can see that a select statement is generate with select all the columns FROM as the _CV.
I cannot post the SQL as it is my code but you can test the same.
suppose 2 records are not loaded to the target. Then in that case we can compare the CV generated by flowgraph with the target table and see what is missing.
Once the flowgraph is created we can call the fowgraph inside a store procedure call with START TASK by passing the parameters.
Also we can see the flowgraph executions on M_TASKS table inside HANA DB and also can see where in the transformation HANA flowgraphs is consuming what time and what kind of SQL is generated at each level.
you can refer to my blog post
https://blogs.sap.com/2022/03/03/sap-hana-xsa-find-invalidated-viewsfunctionsprocedures-and-handy-qu...
is n't it powerful guys ?
Please test it and let me know your views in comments section at the end of the blog.
Thank You
Narasingha
- SAP Managed Tags:
2 Comments
