Agile Data Warehousing with SAP BW/4HANA

The following is a partner blog by Jan Van Ansem, Principal Consultant at Bluefin Solutions.

Traditionalists believe that it isn’t possible to deliver value to the business through the agile methodology on enterprise data warehousing (EDW) projects. However, we may still be riding around on a horse and cart if we didn’t challenge ‘the accepted’.  I’m not only going to challenge, but also illustrate how moving to agile will increase user adoption and ultimately the success of the EDW projects.

Agile software development has quickly proven its value in delivering high quality applications that meet customer needs. When it comes to meeting the customer’s needs EDW projects often fail to meet expectations, as research has illustrated time and time again. Interestingly, ‘Agile’ is not widely adapted as a project methodology for EDW projects.

There is a lot of prejudice against the Agile methodology; perfectly captured here in ‘Fearful of Agile’. In EDW, this prejudice is hidden behind the argument that it is not possible to present anything meaningful to the business in the early stages of a project, or in the short ‘sprint’ cycles, as building the data foundation simply takes too long.

In the age of virtual data marts, fast databases and flexible reporting tools this belief no longer holds true. With the assistance of SAP BW/4HANA and SAP Lumira I’ll prove my point…

 Display data quickly and worry about the modelling later

Setting up an EDW takes time:

1. Business rules and data definitions must be agreed throughout the organisation
2. Improving the quality of data is cumbersome
3. Designing truly actionable reports requires a thorough understanding of business objectives, processes and data.

An architected EDW aims to provide a ‘single version of the truth’ in a central repository which is used for all corporate reporting. One of the pitfalls of such a project is that reporting only starts when the data layer is in place.  Waiting until this point means business users do not get to see anything meaningful until weeks, or even months, into the project. By then some decisions might have been made which could be costly to change.

With SAP BW/4HANA, it is very easy to involve the business user at a much earlier stage. Recently I worked across a project where we took a data sample from an old system as flat files and loaded this into SAP HANA. The first samples were small and loaded in a couple of days, however, we soon scaled this up to data dumps containing entire tables with close to a billion rows. We built some crude virtual models on top of the data structures and before the week was over business users could start building reports in Analysis for Office and SAP Lumira.

The models in SAP BW/4HANA were incrementally enriched with formalized data structures, by associating the fields in the Calculation Views with Info Objects – the building blocks of an architected EDW in SAP BW/4HANA. Assumptions about master data relationships and data integrity could immediately be evaluated with business users. Once all the data elements were associated with InfoObjects we started building the structures for physical data storage and loaded the dataset into SAP BW/4HANA.

By following this approach, we not only made sure that business users were fully involved from the start, but we also had a very high success rate of data loading; almost completely avoiding the time-consuming task of reloading data.

 Let the business do the reporting

Business user involved in EDW projects are often quite data savvy. They have advanced Excel skills and often some experience in creating reports in an analytics tool. They rarely need much guidance to work with tools like SAP Lumira or Analysis for Office.

A great way of working with the business users on reporting is by organizing weekly reporting workshops. This provides enormous value to business users:

1. Weekly tasks can be commonly agreed to work on
2. Questions can easily be answered
3. Users can share their experiences e.g. what they did, what they couldn’t (easily) do, and their learnings.

In this environment, the elements which are difficult to do in the reporting tools will easily be identified.  This is where a back-end solution in the EDW system might be required – usually in the form of a virtual or physical Data Mart.

Put the business user in the driving seat of data acquisition

It is impossible to predict what sources of data business users will want to use in the future. Some of the new sources might only be temporarily of interest (for example market research for a specific campaign). However, when a new source is identified, the business users want to see it incorporated in to their reports now.

An easy way to give the business users control of what sources they use is through SAP Lumira’s data blending function. This function allows you to build a report based on different sources (for example SAP BW/4HANA and Excel) and merge or link different datasets. A downside is that the non SAP BW/4HANA dataset is only available in the context of SAP Lumira reporting, and not for other tools which you might have running on the EDW (planning, predictive, other reporting tools).

SAP BW/4HANA offers two mechanisms which allow business users to pull non EDW data into the context of the EDW (within certain restrictions). There is the ‘Workspaces’ feature in the BW application component and the ‘Agile Data Preparation’ function on the SAP HANA platform. Both functions provide the business user more flexibility to use ad-hoc sources and gives IT the ability to set parameters for data acquisition and monitor the usage of resources of non EDW data. When managed correctly, this is a great way to ensure that the business stays engaged with the EDW instead of walking away in frustration and finding their own solutions in shadow BI systems.

Let’s embrace change and make EDW projects successful

It is time for the EDW world to improve its track record and the Agile project methodology will boost the success rate significantly. The traditional excuses for not using Agile are no longer valid.

SAP BW/4HANA provides the ideal platform from which to work in an agile way: business users can interact with the data from day 1, have flexible tools to do their own reporting, and can be in charge of their own data acquisition, within the parameters set by IT.  What’s not to like? Go forth and be agile!

See the demo at SAPPHIRE NOW

Attending SAPPHIRE NOW? Be sure to view the SAP BW/4HANA as an Agile Data Warehouse demo, created by Bluefin and Lenovo. The demo combines SAP BW/4HANA with the latest SAP Lumira technology to illustrate how organizations can now respond to analytic needs faster and better than ever before.  See it in action at the Lenovo booth (#811).