Let me rewind to late 90s. If you were involved in BI at the time – you would have heard the pitch “create a data warehouse, and 10 years from now – you will have incredible insight”. Sure enough many companies invested heavily in BI. And through thick or thin – the C suite has supported investment in BI, awaiting the promised benefits that will take their companies to greater heights. Now lets fast forward to 2010 – several companies have huge data warehouses now. But are they getting these big benefits they thought they will get?
And now, instead of showing the value of all this accumulated data – we just jumped into the 2.0 world. Now our argument is “old data is stale data”, and only real time data matters. We also say things like “we need a google like interface” as opposed to clunky old GUI that BI traditionally gave. We of course want to combine social media into everything else – so yes, “unless you know at real time what people are talking about you on twitter, you are bound to take bad decisions”.
I am just surprised at how many companies drink this koolaid instead of screaming “I can’t take it anymore !”.
The whole idea of BI is the theory that good data leads to good decisions. This sounds very logical – except this is not always true in real life. There is an abundance of information on internet, and powerful search engines like google. Does that mean everyone analyzes this information and take educated decisions all the time? It is the same in the corporate world. People take bad decisions because of a variety of reasons – and data is just one part of it. Along with investing in BI systems, corporate employees must get training and experience in good decision making. Otherwise – use your BI investment money elsewhere for hedging against impact of bad decisions.
BI does not solve the data quality issues. A hundred reports pushed at you at real time is of very little use if it is based on poor quality of data. Also, information overload is rarely the answer.
BI 2.0 seem to say that systems should aim to interpret data and pass it quickly to operational staff so that they can make better decisions. I don’t disagree with this – except that in real life projects, very few have a good grip on even the less complex world of static data. I don’t see this group of people suddently embracing the 2.0 world and get smart on handling a more complex paradigm. As they say – you must learn to walk before you learn to run.
In my opinion, Real time is kind of over rated. Plenty of decisions can be made with data that is not up to the second. And the need for real time data is not constant – there are certain times when you absolutely need it – like say at financial close. But that does not mean that every other day, information posted in your GL needs to be available in your BI system in two seconds. Yet, many companies take this all or nothing approach to real time data, usually at significant cost. Like with SAP’s NewDB, soon there won’t be a necessity to have a stand alone BI system. Till such time – we can survive by focussing on solving the really critical real time requirements. For example – by throwing additional processing power and memory at it – although not elegant, some of the inefficiencies of reporting out of OLTP systems can be overcome till something like NewDB becomes mainstream. But the purists tend to bristle at the idea.
User interfaces – this is also made out to be a bigger deal than it needs to be. Is a google like interface, the answer to your problems? In my opinion – at best, it is a start for a small section of users. We have to let go of this notion that business users are not capable of handling complex Applications. Nothing could be farther from truth. Most business users I know are very smart and capable of doing pretty much anything that their technical counterparts can do. It is an insult to their intelligence if we think they are not. In every BI strategy engagement I have taken part – I hear that usability is a big deal on the first day. And as I dig deeper, it becomes very clear that the UI has very little to do with it really – it is more about their frustration in understanding what data is available in the system, how current it is, where the data originates, the meaning of certain key figures, performance and so on. IT usually thinks that since they have cleverly modelled everything – it must be the UI that the business is complaining about.
Context is critical to make use of data. I already ranted on it here – http://andvijaysays.wordpress.com/2010/05/26/context-is-everything/ . This is a 2.0 concept and I think at some point we will get there. But we should also understand that there is a limit to what a computer can tell us. A shipping clerk might see a storm warning on TV and adjust his schedules for the day. Theoretically, a computer system can also get this information from some place and make such adjustments. The question is – is it worth the investment to make a system that will cater to an occasional event? That is the thing about event driven BI – we can only optimally model events that we can guess before hand. For the rest, we need better trained users to take decisions. I am not anti-progress. I just think that the whole 2.0 thing is a little beyond what most companies need today.
So my POV : Most companies have everything they need to take good decision making. More tools or fancier acronyms is not what is going to get them to make better decisions. What they need is to go back to drawing board – IT and Business together – and figure out a way to make better use of what they have. Check if you have an up to date BI strategy, and if not – make an effort to decide on your strategy first. Check if your users know what information is available to them and how to access it. And please do some housekeeping on BI . I am sure you will find the hidden gems without throwing away a lot of money. Once you do all of this – let’s usher in 2.0 or 3.0 or whatever else is the popular flavor at the time.