Is your BW the Data Warehouse, anyway?
There seems to be a big wall between the worlds of BW and “traditional EDW” (whatever it means). Go to the LinkedIn forum of The Data Warehouse Institute (TDWI) and try to open a mouth that you are working with BW – you will be smashed immediately with many comments like “[BW] has always been an ancient, clunky and monstrous solution to implement.”, “I’m yet to come across an SAP customer that isn’t into their Nth attempt at trying to get it to work properly or efficiently.” or “How much did [BW implementation] cost (software, hardware, consulting, etc.), how long did it take and how much frustration did the business users feel while waiting for a solution?”
Same when I talk to my data warehousing colleagues specialized in traditional BI/DW/ETL tools: the only things they want to hear from me are “What are weak points of BW?”, “Why performance in BW is so bad?” and “How can we replace BW with ERWin/Informatica/Netizza/Microstrategy (or any other set of tools)?”.
I had quite many discussions like these on forums and face-to-face, and I had always been on the position to defence BW as a rich, powerful, open and flexible data warehousing platform. But here at SDN I want to openly ask my BW fellows: “Is your BW the Data Warehouse truly?”. Far too often I found that it is not, and here are my thoughts why:
- Roots are in the history of BW as a reporting extension to core R/3. Let’s have a look at the BW purpose as defined for release 3.0B: “The SAP Business Information Warehouse allows you to analyze data from operative SAP applications as well as all other business applications and external data sources such as databases, online services and the Internet.” Yes, it comes with powerful OLAP engine, but no one talked about BW as a data warehouse at that time; it came later, but it’s true meaning is still not recognized.
- IMHO many BW application developers lack data warehousing education. Names like Kimball or Inman are just an empty sound for them. I hoped that Arun Varadarajan would develop discussion that he had Are you an Inmonite or a Kimbalite ?, but the discussion had stopped before even started. Most of BW developers are coming from ABAP, sometimes from other SAP functional area (“BW is hot on the job market!”) or directly from schools. While knowledge of programing language (i.e. ABAP) is much more crucial in BW comparing to other market DW tools and knowledge of FI, CRM or SD source tables is extremely helpful, many BW people have How many skills does BW consultant need? in the data warehouse architecturing, DW data modeling or data quality implementation.
- To previous point I would add a problem of belief in the “Best practices”. Often I had been asked about best practices in specific BW topic, and my reply always was “What are your requirements?”. Still people prefer to hear that there are magic out-of-the-box one-size-ftis-all tricks that turn their BW into the robust data warehouse overnight. I saw when it was enough that consultant said “It is the best practice to code all transformations as an expert routines” and the statement was taken as rock solid (real-life example).
- Unfortunately SAP does not provide many examples of good data warehousing structures either. BI Content is simply not following data warehousing principles: not designed for performance, not designed for multi-layers architecture, not designed for integration of data from multiple source systems. Even much promoted SAP Layered Scalable Architecture (which I am a big fan of) is not reflected in SAP-delivered BW objects from Business Content.
- Another problem is that there is no SAP methodology for BW implementations. ASAP was not the right methodology for BW and trying to deliver BW projects accordingly to it was like trying to fit a cube into a round hole. I have hoped that after acquisition of Business Objects SAP will develop BO’s “The BI Solution Accelerator” to the next level as SAP’s own BI/DW implementation methodology, but I do not see this happening so far.
- Anywhere else I haven’t seen such a big disconnect between two groups: application developers (i.e. BW apps developers) and system administrators (i.e. Basis) as it is in the world of BW. It is getting even worse when customers are outsourcing BW development to one company and Basis to another. To squeeze optimal performance from BW you need these two groups work together in constant dialog right from the beginning. My favorite example is when BW consultants are tuning OLAP Cache in RSRCACHE to 200MB, while Basis keeps system parameter for available export/import memory in RZ11 at default 4MB. And untill these two groups start talking to each other, BW data warehouse performance will suffer giving us bad publicity in the outside world.
Net, it is as well up to us to fix the perception of BW as non-relevant data warehouse in the world outside SAP and at the same time to join and influence that EDW world as an equal partner.