Skip to Content

Over the past few months, I’ve presented on this topic to many customers and colleagues. As there seems to be such a high demand, I’ve decided to convert the underlying slide presentation into two blogs, with HANA and BW 7.30 – Part 1 focusing on the motivation, scenarios and use cases while the second looks at the combination of HANA and BW from a technical angle. Before I continue with the second blog please note that the usual disclaimer applies. 

Evolving the In-Memory Footprint in SAP BW

Let’s first look back to 2003 when the in-memory efforts were started within BW and have led to BWA (aka HPBI, HPA, BIA) Pushing BI To A New Frontier. Understanding the evolution will make you understand the rationale also on future developments.

Figure 1 pictures a matrix listing various layers of BW in the rows and the recent releases in the columns. You can see that initial investments have been around those areas which were natural first targets, namely the SQL processing behind infocube based queries. With BWA 7.2 this could be enhanced to multiproviders and there is even de facto support of DSOs via the hybrid provider. See this document for more details on the BW 7.3 / BWA 7.2 combination. Looking beyond the latter, I will show some examples of the in-memory impact on the planning engine and the data warehousing layer below.

Evolving In-Memory Footprint in SAP BW
Figure 1: Evolving In-Memory Footprint in SAP BW

RDBMS + BWA Becomes One HANA Box

There is certain advantages of moving from a BW setup based on a classic RDBMS server complemented by BWA to a BW system sitting on top of a single HANA server that has the combined abilities. From a technical perspective, the single HANA instance removes the necessity to manage consistency across two servers (as with an RDBMS and a BWA). This is particularly interesting for planning (write-back) scenarios and simplifies matters a lot. From an admin perspective, it is more ambiguous: some BW customers like the separation that the RDBMS caters for the warehousing while the BWA for the querying load respectively. On the other hand, two servers, hardware installations, licenses need to be maintained.

In-Memory Planning

Let’s turn to a category of features that goes beyond the traditional outstanding query performance advantages of HANA or BWA respectively, namely the evolution of BW-IP into in-memory.

Figure 2 shows a “Hello World” example for a planning operation: a set of cells are displayed to an end user who decides to increase one of the values from 250 to 300. What happens now when this change is submitted to the server? Remember that planning is typically done on an aggregated granularity (here: countries and years) while the data is much more detailed – in the example of figure 2: countries break down to branches, years to weeks. This means that changing a single value in the UI frequently translates into a large number of changes on the data level. This is called disaggregation and constitutes a frequent operation in a planning context.

In the traditional approach – meaning that processing is mostly done in the application server – the delta of the change (here: increase by 50) is calculated, then that delta is broken down to the actual data granularity (here: 52 weeks in 2011 and 500 branche in Germany) resulting in a potentially large number of values (here: 52 * 500 = 26000) that are then sent over a network to the RDBMS to be saved.

In the in-memory based approach, processing is pushed down to HANA, i.e. close to the data, by turning around steps 2 and 3, i.e. only a single value (here: 50) is sent to the DB engine accompanied by the disaggregation instruction and its associated parameters. Performance gains in that approach originate in the following:

  • reduced network traffic (1 value being sent to the DB rather than 26000)
  • operations in the DB engine are implemented directly on the data structures existing in that layer, i.e. it is not necessary to convert or cast
  • disaggregation can be highly parallelized

This example follows a very generic pattern that can be applied in many other areas too. Some colleagues use terms like data shipping(traditional approach) vs function shipping (in-memory approach).

A 'Hello World' planning example
Figure 2: Comparing the traditional vs the in-memory approach of a “Hello World” planning example.

In-Memory DSO

The pattern shown in the in-memory planning example, namely (a) to avoid sending huge amounts of data between application and DB servers, and (b) to implement performanc-critical operations directly on the engine-based data structures, can be applied to the BW data store object (DSO) too. To that end, it makes sense to recall how a DSO works (conceptually) and where performance becomes critical. Figure 3 shows how a DSO works:

  • There are 3 data containers:
    • one for the current data – current image,
    • one for the recently uploaded data – future image, and
    • one for the result when the differences between the current and the future images are calculated – delta image, the result of data activation.
  • In traditional RDBMS, these 3 containers are represented as a table respectively.
  • There are 4 fundamental operations on a DSO:
    • upload new data,
    • query the current data,
    • read the delta data,
    • activate – see above.

In a traditional RDBMS environment, querying and data activation are performance critical. Now and as we know, querying is a fundamental strength of HANA and is not of concern anymore. However, data activation needs to be carefully considered: reconciling the current and the future images typically translates into moving lots of data from the DB to an application server where the data is matched and the delta gets calculated. The data activation is a clear candidate to be pushed down into the HANA engine.

Traditional DSO
Figure 3: Traditional DSO

The DSO has been natively implemented in HANA. Figure 4 basically indicates that it has been implemented as a black box that behaves as the traditional DSO shown in figure 3 and described above. The queryingand delta read operations become logical views on top of the black box. The upload operation is straightforward. The data activationhas been natively implemented and shows significant performance gains. As indicated, the sources of those gains are less data traffic and implementing it natively.

In-Memory DSO
Figure 4: In-Memory DSO

Conclusion

This concludes this second part. Hopefully, we will be able to write about and discuss more details in the course of the next months. However, the examples above should provide a good flavour of what is possible and what can be expected.

To report this post you need to login first.

4 Comments

You must be Logged on to comment or reply to a post.

  1. Gregory Misiorek
    Hi Thomas,

    i think your point 3 is the most compelling to the business user. if we can visualize 26,000 records being sent across network as the current load vs 26,000 records processed in memory yielding substantial savings in response time this may be something end users can relate to. today, they are literally waiting for another refresh from the database to see the update in their spreadsheet.
    what’s more we are no longer restricted only to planned data, but any data, including actuals, as they become available from the transactional engine.

    rgds,

    gm

    (0) 
  2. Shehryar Humayun
    Hi Thomas,

    Thanks for insightful blogs. These are quite useful to understand HANA positioning and features.

    In the podcast you did with John Reed in June, Podcast: Straight Talk about HANA with SAP, there was some discussion on HANA potentially impacting traditional data warehousing concepts. It was clarified that the concepts should hold, the realization might change.

    Would it be possible for you to blog on what impact HANA will have in an LSA based implementation of SAP BW? For example:

    1. Given that HANA is currently delivered as an appliance with Sybase and DataServices for data acquisition, should these components become part of LSA Data Acquisition Layer, or would you rather treat these as source systems and only consider DSO/PSA as part of acquisition layer?

    2. Similarly, in addition to Enterprise DW and Architected DW layers, you highlighted previously that Agile Data Marts are a good use case for HANA. Should this be reflected in LSA?

    So a blog along these lines which talks about LSA realization in the new HANA world would be great.

    Thanks and regards,

    Shehryar

    (0) 
    1. Thomas Zurek Post author
      Hi Shehryar,
      good points. What you raise are clear talking points within the team. Once those discussion converge I’m happy to blog about them. But there is still too many floating items. However, in the meantime, it is certainly worthwhile to get outside thoughts!
      Regards
      Thomas
      (0) 
  3. Witalij Rudnicki
    Hello Thomas. Unfortunately we did not get much time to talk during TechEd Madrid. Let me ask my question here then.

    What in-memory calculation scenarios are supported with current release of BW powered by HANA? Same as with BWA 7.2, less, or more?

    Thank you,
    -Vitaliy

    (0) 

Leave a Reply