Skip to Content

Prologue

Previous parts on SDN:

Apologies for the delay in posting part 3. Reason: bock beer season!! (http://www.beeradvocate.com/beer/style/32)

 

Miscellaneous Topics

I like miscellaneous topics. They are so … miscellaneous.

Right. Enough introduction. Let’s get on with it.

 

Apologies to Support Consultants & HANA to the Rescue

In Part 1, I described the formation of “data holes” in BW cubes and more or less suggested that support consultants are to blame for these holes. My apologies. I am an implementation consultant. Blaming support consultants is like second nature to me. But sometimes, implementation consultants screw up as well.

Below an example in which an anonymous implementation consultant (ok, I confess … it was me) built a wrong solution for data lookup from another DSO. A certain set of fields in the left dataflow is read from the right data flow. The solution assumes that a value change for any field within this set triggers not only the right data flow but also the left data flow.

An existing solution that worked properly – assumption was apparently valid – was enhanced with an additional field for which the assumption was not always valid. Subtle issues like this are usually not found during testing, but are noticed after go-live. This is a typical example of the type of data integrity issues found all through a traditional data warehouse. Fixing the data issue involves carrying out some full loads instead of delta loads.

And this is where HANA technology can come to the rescue. In a traditional database, one would think twice before carrying out a data load involving millions of records in full mode on a daily basis. With the speed of HANA technology this becomes an option seriously to be considered. Below a solution quite successful in practice, i.e. a “self-update in full mode”. All lookup actions are carried out in a transformation from a DSO to itself in full mode.

No delta-trigger is required to pick up new values. Preferably, the self-update transformation is kept “ABAP-free” for optimal performance. Application of the “self-update in full mode” solution for all lookup actions can and does strongly increase data integrity in a traditional data warehouse.

 

Convergence of BW objects

Below, SAP’s standard graph displaying the convergence of BW objects moving towards BW on HANA versions. Only one object for data persistency remains, i.e. the “Advanced Data Store Object” or ADSO. The Open ODS View and Composite Provider are virtual objects, which do not even require the use of InfoObjects. InfoObjects remain, but will in future probably only be used in case of complex master (hierarchies, authorizations, multi-language descriptions, …).

This convergence is a good thing. At first sight, it may look like functionality is disappearing, but all options existing in the separate objects on the left side appear as settings on the limited number of objects on the right side. These new objects are thereby much more flexible than the old ones. A type of object cannot be changed after creation, but a setting on an object can.

The same tendency is noticed in Native HANA objects. Attribute Views and Analytical Views are replaced by more flexibility in Calculation Views. SAP itself is using only Calculation Views in its Hana Live offering, and most customers are currently doing the same.

Hurray for SAP!

 

Convergence of Frontend Tools

An even better idea was the convergence of frontend tools. The overall idea is sketched in the graph below (not the latest version, but this version most clearly demonstrates the convergence).

After takeover of Business Objects by SAP, a huge number of reporting tools was scattered all over SAP’s product portfolio confusing not only BI veterans like me, but also most customers, who decided to postpone investments in reporting tools or look outside of the SAP portfolio. It was a necessity that SAP would make a statement like “These are the frontend tools for the future. We will continue to invest in these tools.” Luckily they did, although a bit late. The strongest and most future-proof tools – Analysis Ofice, Lumira and Design Studio – were all new developments by a joint team of SAP and Business Objects developers after the takeover, and not attempts to integrate mature Business Objects products with SAP technology, which for some reason does not work well. For more details, please read the blog on this topic, which is also the source of the graph shown above: http://blogs.sap.com/analytics/2014/06/25/run-simple-convergence-of-the-sap-businessobjects-bi-product-portfolio/.

More convergence: Lumira and Design Studio are joined in next releases. On the downside: the cloud version of Lumira is recently replaced by Cloud for Analytics aka Business Objects Cloud, a new product. Are we diverging again? If not, hurray for SAP!!!

 

Everything in one HANA box?

In Part 1 I stated at some point that HANA technology makes it possible to put transactions and reporting back together in one box. Somewhere else I also stated that data storage is cheap. The second statement clearly is not true in case of HANA in-memory storage. Putting everything in one box may be feasible for smaller companies, but larger companies will rarely choose to do this. System architectures arising will include multiple something (servers, clients, tenants …), often to facilitate “Multi-Temperature Data Management”. See the SAP-graph below for an explanation.

The client I currently work for has a Native HANA analytics system on a database which hardly contains any data, only virtual models. The data is kept in separate Operational Data Stores, in most cases also HANA databases, connected to the analytics system via Smart Data Access. SAP source systems also run on HANA databases. SAP source system data in the Operational Data Stores is synchronized with minimal delay using System Landscape Transformation. This type of setup will be seen often in practice for larger companies. But if you want … you can put everything in one box.

There is a lot more to say regarding system architectures for Native HANA, BW on HANA and hybrid solutions. One of my colleagues is currently working on a blog on this specific topic.

 

Skills: from ABAP to SQL

In the (near) future, SAP BI consultants will need new skills: less ABAP and more SQL. In Native HANA, enhancements can be built with “ABAP Managed Procedures”. Don’t let the name fool you: there is no ABAP involved in this type of enhancement, only SQL. HANA operation of BW transformation is already disabled after pressing the Start/End routine button alone. Any coding involving something like “loop at datapackage” is by definition row-based and not compatible with high-speed HANA-operation. Also, frontend tools like Design Studio require more SQL expertise than before.

SQL brains are needed instead of ABAP brains. Advice: In a team of 4 SAP BI consultants, fire 1 and replace him/her by someone with traditional database skills, someone not spoiled by the fancy development shells SAP is providing. There are plenty of them around in the non-SAP world. They are usually quite cheap.

 

How about Historical Data?

The openSAP training series introducing S/4HANA were quite shocking for BW consultants, as BW was not mentioned once during the entire series. With disbelief we watched the lesson on “Data Footprint Reduction” by Wieland Schreiner, who for some reason wanted to shrink the database of an ERP system in such a way that it would fit in an iPhone. Why? Don’t ask me. I am not Wieland Schreiner. The first steps I could relate to: compression, column store, remove redundant tables, all fine by me. But in a “grande finale” Wieland Schreiner said, and I quote: “If we now take out of that system all that is not needed, so all historical informations …”.

There he lost me. Maybe there are organizations living so much in the moment that they are not interested in data older than, say, a year, but personally I have not encountered such organizations. This is where some sort of data persistency is required in the data integration platform: for historical data that is not kept in the source systems anymore. But also for “snapshots” or frozen states of data that are either not stored in the source systems, or only in obscure logfiles. For this purpose, a traditional data warehouse system like BW is an obvious choice, although one can think of alternatives like physical tables in the HANA database created with HANA Studio and loaded with data using Data Services. In recent presentations on BW/4HANA, SAP is positioning BW for historical data (and historical data only).

Which is an excellent bridge to the final miscellaneous topic …

 

What is BW/4HANA?

BW/4HANA is the “new-objects-only” version of BW on HANA.

 

What is BW/4HANA? – Revisited

Well, maybe that was a bit short, although it sort of covers it. What does one do if one wants to get rid of old technology hampering future developments, like cubes and stuff? A two-step approach is required. First, one releases a version with old and new technology side-by-side. Second, one releases a version with new technology only. The side-by-side version is supported till the end of time, but all innovations are focusing on the new technology. In time, all clients will need to migrate to the new-objects-only version.

How easy will it be to migrate from BW on HANA to BW/4HANA? Let’s see what SAP itself says about it in its BW/4HANA FAQ document: Although, SAP BW/4HANA is not a legal successor of SAP BW (based on SAP NetWeaver), a conversion path will be provided.. Oh boy, SAP is going into “legal-mode”. Be afraid, be very afraid! Here is another quote: Classic objects in BW-on-HANA can be semi-automatically migrated into SAP BW/4HANA compliant objects.. Ouch. Translation: “It will hurt”. However, nothing a good implementation partner won’t be able to help you through.

Finally: Say goodbye to the good old “Business Explorer” frontend. But let’s be honest: wasn’t that about time?

 

Epilogue

Feedback from one of my colleagues on the draft version of this blog: “In parts 1 and 2 you gave your opinion a lot. In part 3 you are more ruminating SAP slides.” He may be right. Therefore, in a nutshell, my opinion on today’s topics:

  • Frontend tools: Stop using the old SAP stuff. Focus on Lumira, Design Studio and Analysis for Excel. Do not underestimate how much report users like Excel and the amount of self-service it provides them. In a situation with predominantly SAP source systems, stop using QlikView, Tableau and other non-SAP tools. These tools may currently have some benefits in functionality, but integration with the backend is worse and in the long run more important.
  • System architecture: To me it seems that the customer I currently work for is on the right track with its approach. Separate servers for analytics and for data storage combined with Smart Data Access technology provide a lot of flexibility. Still some child diseases, but it makes sense for the future.
  • BW versions: Convergence of BW objects is an improvement, but is by no means revolutionary. You can stick to your older BW version for a couple of years more. BW/4HANA only for customers that are new to BW.
  • Positioning of BW: As a BW-veteran it hurts to admit that the golden years of BW – if there ever were any – appear to be over. For the future, I agree with SAP that BW should only be considered for historical data. But even there Data Services with persistency in a HANA database is a strong alternative.

This blog and the two previous ones more or less wrap up what I have to say about the topic in the title. Please contact me if you would like to cheer, disagree, complement or chat à freek.keijzer@ciber.nl.

To report this post you need to login first.

5 Comments

You must be Logged on to comment or reply to a post.

  1. Donnie Burhan

    Hi,

    I got some questions or maybe doubts about all these new things that SAP has been spouting recently.

    1. SAP has been pushing for Calculation View instead of Attribute and Analytical Views. I think we got a pretty heated old thread where some SAP guys just say something like, “Don’t ask, just do it” when asked why only use Calculation View instead of utilizing Attribute and Analytical View. But as far as I know, HANA got OLAP engine which is only utilized by Analytical View. Now, is it wise to calculate aggregation without the OLAP engine?  How far can we abuse the brute force of HANA, depending on simplicity without taking design and performance into thoughts?

    2. We still need to use InfoObjects for Navigation Attributes, Time Dependant, and Variable functionality. Navigation attributes is a comfort zone to me that I will ask questions as to why someone would do multiple joins in Composite Provider for master data lookup which will make it confusing to look at. Time Dependency is too complex to code in ABAP or SQL. Field-based modeling is also subject to abuse, especially for the BEx variables. I’ve seen multiple times when people create BEx variables based on the Field, then they need to make multiple variables with the same logic for multiple fields which basically the same thing.

    3. When I heard Lumira and Design Studio become a single product, I was kind of happy. But then it’s revealed that actually it’s only sharing Lumira name (Lumira Discovery and Lumira Designer) while it’s still two different product with two different license model, then I’m speechless.

    4. BW only for Historical data gives me some laughs. Maybe they can say it again when the new tools are already mature enough that it can handle time-dependent data, lookup on million of rows of non-persistent data in memory. Maybe by the introduction of S4/HANA it’s possible, because it will transform how the transaction data stored to be easier for reporting. But until when the customers need to wait for all modules to complete the transformation? Also let’s not go into the Industry Specific solutions, cause the situation is more severe over there.

    But yeah, maybe it’s time to polish our SQL skills because SAP sure starting to push BW to the edge while they actually still depends on them. But I think visual tool is still better to use because of less efforts for maintenance and reading other people’s code hasn’t gotten easier even now. ALV reports based on CDS view getting data from Analytical/Calculation Views should be ideal for operational reporting and lift some weights off our shoulder.

    (1) 
    1. Freek Keijzer Post author

       

      Donnie,

      I am honored by your comment, which is more like a blog in response to my blog. I think I can let most of your statements speak for themselves, and I can agree to the bigger part of them.

      Some points I would like to clarify to you and other readers:

      • Let there be no doubt that I am pro-BW. 15 years of my ICT-experience has been predominantly with BW. If there were no BW-work tomorrow, I would be in big trouble. But 15 years of experience with BW also led me to the conclusion that BW – or any other data warehouse for that matter – is not working in a complex environment. Users expect the basic data to be the same as in the source systems, and with all the data loading going around, BW simply cannot deliver to this expectation. In complex environments, users hate BW. Virtualization solves this problem.
      • … and of course I agree that Native Hana and virtual BW still have a long way to go. InfoObjects are indeed still handy. But I agree with SAP’s long-term vision that BW – or any other method of data persistency – will only be used when unavoidable, e.g. historic data.
      • Licensing … I do not know much about. But I get the impression that no one does anymore. Luckily, I am not the type of consultant that needs to explain SAP’s licensing strategy to the customers.

      Regards,   Freek

      (0) 
    2. Marc Bernard

      Hi Donnie,

      just a few comments that might help understand:

      1. Focusing on one engine simplifies things and allows us to focus on one area for performance optimizations. And in the end, no one cares how many engines are under the hood.
      2. InfoObjects are indeed very important and provide many services that you can’t get with field base modeling. You can, however, use Open ODS Views also for navigational attributes (nav fields so to say) by associating them to your data models (CompositeProviders).
      3. Licensing is usually a bit behind the simplification and consolidation of products. We understand the complexity and are working to simplify on the licensing front as well. Some times it just takes a bit longer. Legal things, you know 😉
      4. See my “PS” to Freek below.

      With the introduction of SAP BW/4HANA is should be very clear that we re not “pushing BW to the edge”. To the contrary, data warehousing is now back on everyone’s agenda again.

      Best,
      Marc
      Product Management SAP HANA DW

      (0) 
      1. Donnie Burhan

        Hi Marc,

         

        Thanks for the positive response.

        1. Yes, no one cares about how many engines under the hood, if the performance speaks for itself. I don’t see any updates on this topics anymore. SAP recommends to only use Calculation views without giving good reasons. And the last time people are having the discussion, can’t say that we are having good response. Performance wise, it seems there has been some improvement, but still breaking down on more complex model. And now it seems SAP is crazy about their new CDS, and they can be utilized with RSRT too. Honestly, I’m quite lost on what is the best direction to move on. Could you advise on what would integrate best into the BW landscape, or maybe use case which to use for certain situations?
        2. Yeah, I just hope some developers will think once again before recklessly using the “easy way” which would risk the maintenance in the long run.
        3. I can only hope for the best.
        4. Again, I can only hope for the best.

        I hope that SAP will make BW great again.

         

        (0) 
  2. Marc Bernard

    Hi Freek,

    first of all congratulations and thanks for writing an excellent 3-part blog. We appreciate your opinions and feedback a lot and it’s important to hear how customers and consultants are adopting the innovations we deliver.

    A few comments from my side:

    “BW/4HANA only for customers that are new to BW.” That is certainly not how we position it. SAP BW/4HANA provides great value for all existing BW customers whether they are still on other database platforms or already on SAP HANA. The simplification that customers can achieve by transitioning to SAP BW/4HANA is tremendous. Much of this you describe yourself but how many customers have done the switch to Advanced DSOs, Open ODS Views, and CompositeProviders for their complete BW system? How many have reduced the layers? Who many are struggling with data life-cycle management and dream of automating this “no business value” task? And who has really done integration with Big Data or tapped into logical data warehousing?

    When customers are relatively “far away from SAP BW/4HANA” and would benefit a lot from it, justifying the investment. When customers are relatively “close to SAP BW/4HANA”, for example if they started on SAP BW powered by SAP HANA and are using SAP BusinessObjects exclusively, then the investment to get to SAP BW/4HANA is relatively low and the transition is pretty easy putting these customers quickly back on the innovation track.

    Anyway, lot’s of words and we need to prove that this plays out in projects as well. I have been a BW consultant for much of my SAP career and I’m very confident that customers will love SAP BW/4HANA. Hopefully, you will find a project where you can give it a go soon!

    Thanks again and I’m looking forward to more blogs from you,

    Best,

    Marc

    Product Management SAP HANA DW

    PS: Modern Data Warehousing is certainly not just for historical data. That’s an over simplification (and a flaw on marketing slides). For example, one of the reasons data warehousing even exists is to be independent of the source systems since they are not available, reliable, and fast enough especially if you aggregate across all your sources. Until customers have their complete landscape for example on SAP HANA and full master data management implemented across all their systems, I’m sure data warehouses will be loaded continuously with all data – not just the history.

    (0) 

Leave a Reply