Standardization is a key aspect of SAP BW Layered, Scalable Architecture (LSA), SAP’s best practice in Enterprise Data Warehousing. One of the ways to realize standardization in the data staging process is using Data Flow Templates.

SAP BW release 7.3 introduced a new modeling object called Data Flow. A Data Flow acts as a container for storing the data modeling objects of a data flow, e.g. InfoProviders, Transformations, DTPs, etc. It can also be used to incorporate documentation belonging to its data modeling objects. Furthermore, it’s possible to define customized / tailor-made Data Flow Templates to facilitate standardization of data flow patterns in the context of your SAP BW implementation and architecture guidelines. Please refer to SAP Help for more information on Graphical Modeling, Data Flows and Data Flow Templates.

In this blog I would like to discuss standardizing data flow patterns using Data Flow Templates, creating new Data Flows based on such a Data Flow Template and the advantages of this approach.

Data Flow Templates

The purpose of Data Flow Template is standardization of data flow patterns. Every Data Flow should be based on a Data Flow Template. From an architecture point-of-view, any deviation from Data Flow Templates should be justified and motivated. It can potentially identify the need for an additional Data Flow Template, to be decided upon by the responsible person or team.

An example of a Data Flow Template can be found in the next screenshot.

Figure_1_Example_Data_Flow_Template.jpg

Figure 1: Example of Data Flow Template

The data modeling objects are represented by the blocks. These blocks act as place holders for the future data modeling objects which can either be created from scratch or reused as an already existing object. The technical name gives an implementing hint for the proper naming convention to be applied.

You can also store documentation. In the context of Data Flow Templates, you can find here modeling tips and procedural aspects.

The following screenshot shows an LSA compliant example implementation with Data Flow Templates Please note a strict segregation between Data Warehouse Layer and Data Mart Layer.

Figure_2_Data_Flow_Templates.jpg

Figure 2: Data Flow Templates

Data Flows

You create a new Data Flow in an appropriate InfoArea. Here you will see a blank canvas where you have to insert a Data Flow Template.

Figure_3_Create_New_Data_Flow.jpg

Figure 3: Create new Data Flow

All data modeling objects of the Data Flow Template are copied into the new Data Flow as place holders. From here you can either create the data modeling objects from scratch or reuse already existing data modeling objects.

Figure_4_New_Data_Flow_with_Place_Holders.jpg

Figure 4: New Data Flow with place holders

At any point in time you can use the function Complete Data Flow to add already existing additional objects, such as DTPs, Transformations and InfoPackages. This is usually also necessary for SPO to complete the Data Flow with all objects related to the SPO.

Figure_5_Complete_Data_Flow.jpg

Figure 5: Complete Data Flow

You can complete the Data Flow in an incremental way until it’s finished. Don’t forget to add any interesting or crucial support information using the documentation feature.

Figure_6_Incremental_Completion_of_Data_Flow.jpg

Figure 6: Incremental completion of Data Flow

Conclusion

In this blog I presented a way to facilitating your (Enterprise) Data Warehouse Architecture by standardizing data flow patterns using Data Flow Templates. In my opinion it’s easy to use but very powerful functionality. I can highly recommend using Data Flow Templates in order to not only increase standardization of data flow patterns but also to provide guided implementation with documentation of necessary steps and naming convention hints. Furthermore, Data Flows can help reducing the need for a complex InfoArea and Application Component Hierarchy. Another benefit is the documentation feature to incorporate on-line documentation of your Data Flows. Last but not least, Data Flows offer a great help in collecting the right data modeling objects using the Transport Connection.

To report this post you need to login first.

5 Comments

You must be Logged on to comment or reply to a post.

  1. Benedict Venmani Felix

    hi Sander,

    I had to oppurtunity to work in one of the projects that you architected. And I should say that I was amazed by how a properly structured BW looks and how it was easy to manage one.

    But, I have a question. In a BW system with huge data volumes and strict timelines for reporting data(time between extraction to cube load), how can you achieve a balance between a structured flow and one that does not have many layers, having more than two/three layers delays yourtime lines?

    Thanks,

    Benedict

    (0) 
    1. Sander van Willigen Post author

      Hi Benedict,

      Thanks for your feedback and nice hearing from you. I would like to react on your question.

      Consider structuring, organizing and standardizing your (Enterprise) Data Warehouse as the starting point: a baseline which is in my opinion definitely a prerequisite to be successful and necessary to survive on the long term. It does not have to be conflicting with scalability or performance. I would say the contrary is true.
      First of all you must design your Data Flows as lean as possible. This applies especially to persistent storage of data. Respecting one of main architecture principles is decoupling the Data Warehouse Layer from the Data Mart Layer. The minimum persistent storage is once in the Data Warehouse Layer (i.e. Propagation Layer in LSA) and once in the Data Mart Layer (i.e. Reporting Layer in LSA). The other layers normally only contain InfoSources and Transformations so no persistent data storage.
      The next recommendation is to introduce scalability by partitioning transactional data. As part of LSA we can distinguish between strategic partitioning (using data domains) and tactical partitioning (normally using time characteristics like calendar year and fiscal year). This can be combined with table partitioning (depending on your database).
      Furthermore, use all possible performance best practices, e.g. no SIDs in Data Warehouse Layer DSOs which can significantly reduce the activation time of standard DSOs.

      In the context of this blog, you can predefine such data flow patterns and implementation standards in your customized Data Flow Templates. Even you can think of dedicated Data Flow Templates for special cases (e.g. high volume data). The bottom line is reducing a multitude of heterogeneous data flow variants to a limited set of standardized data flows patterns using Data Flow Templates.

      Best regards,
      Sander

      (0) 
  2. Henry Jones

    Hi Sander… How do handle table lookups in abap? For example, if I have a table lookup in a start routine, how would I document that in this tool?

    (0) 
    1. Sander van Willigen Post author

      Hi Henry,

      First of all, you can add documentation to the respective Transformation where you do the lookup. Furthermore, you can also include any “reference” BW InfoProvider (usually DSO) in the Data Flow. This will visualize the dependency on this InfoProvider although it is not directly linked in the data staging process.

      Best regards,

      Sander

      (0) 

Leave a Reply