A lot of material has been published in the SAP world about ESOA making various business processes super efficient. Discounting the element of marketing hype, we still have a lot of information to glean about making our transaction processing better by taking the ESOA route. What usually gets sidelined in the ESOA roadmap is the possibilities of taking an SOA approach to Business Intelligence and Analytics.
There are several reasons for this – but in my experience, three things cause this more than anything else.
- Lack of business case for “data services”
- Lack of technical awareness of possibilities in BI, MDM etc for ESOA
- Lack of a unified data model, even at a conceptual level, within the organization
This is an awkward situation – since neither Business Experts not IT experts will make a case for exploring the concept of “data as a service”, and as a consequence they both lose out. I am very careful when I broach this subject – since it usually only succeeds in starting another round of finger pointing between business and IT, and everybody gets real exhausted by the time we stop the blaming process and attempt a stab at the solution.
Since the idea is to make you an ESOA champion, not an ESOA martyr, a few warning shots are in order.
Whatever you do – try not to ask around “hey, do you want some good data around here?” This will quickly earn you the kind of reputation that is hard to lose, however hard you try! Every body can use good data that is available in a timely fashion. If some one says they don’t, stay clear of them; it is not safe to be near a drowning boat.
While we are at it, here is another piece of advice if you care to listen – Aim small, miss small! Try not to go with all guns blazing to make your company an ESOA role model overnight. There are enough ESOA failures around in this world, and there is nothing original in jumping off the bridge because three of your pals did so. Same goes for the unified data model – these things take time. Be pragmatic – and take baby steps.
So, here are a few options to chew on.
1. Not all data is needed physically in one system
Most CIOs like to consolidate their investment into a few systems, with the idea that this will reduce the TCO. Historically, this has been a wise decision for the most part. If you had ten data warehousing systems, you would have to spend a fortune in writing and maintaining ETL, run the risk of data being redundantly stored in multiple systems, pay licensing fee to ten vendors etc.
This paradigm needs to be revisited since technology has moved forward a lot. It is very feasible to have federated data warehouses now. With the advent of ESOA, it is all the more possible to use information spread across several sources.
Here is the general idea. Every data warehouse will publish services to a service bus, and consumers will subscribe as needed. The idea is to report “virtually” as opposed to “physically”. You do away with a lot of costly extraction and loading processes (the E and L of ETL). Transformations might still be required if these datasources have different models. Hence the need for a unified consistent data model goes hand in hand with this scenario.
In case you are not aware, SAP BI does have the capability to expose queries as web services, and you can use variables etc as you do with BEx. Do a search in SDN and you will find the required information. In addition, you can read cubes etc from BI directly through standard delivered RFCs without using a query.
2. Not every user will use the same tool to access the data
There are several great reporting tools out there in the BI market– Cognos, Business Objects, MicroStrategy, Bex etc. Traditional approach has been to tie the front end and back end tools of a particular vendor together (like BEx being the front end of choice for SAP BI). If the reporting tool has an adapter to SAP BI, and most of them do, they can connect without much trouble. But all this comes at a cost.
Once a user gets used to a tool, it is hard to ply it away. Often times, the business is ok with a higher TCO to keep these multiple tools around since the change management is pretty hard if they have to consolidate.
The good news is that almost every vendor out there now has tools that can use webservices. This means that you can keep the tools that make you productive without physically moving data for the sake of reporting them out of a tool of your choice.
There is another aspect that we can consider here – the use of unconventional tools for reporting, like widgets. Widgets can work with webservices, and can play an important role in the ESOA world. Wouldn’t it be great to get weather, sports scores and accounts receivables all on your desktop, without ever logging on to the portal?
3. Not all data is needed at real time
This is an easy trap to fall into. When we go chasing ESOA keeping our transactional systems in mind, some how the instinct is to make everything real time. This is not quite true in the analytics world.
A good part of data that allows us to make decisions are based on historic trends, that do not get significantly affected by the last 5 transactions that happened within the past hour. Remember this before you go all out for things like real time data acquisition in SAP BI.
4. Most of the governance pitfalls of standard software life cycle is applicable to ESOA based analytics too
Let us consider a project in SAP using ABAP as the development platform. Every developer has a different idea of the best way to solve a problem, say getting the address of a business partner – some might use a standard SAP function, another might use an open SQL statement etc. After a while, you end up with several incarnations of function modules that try to solve the same problem. Variants of this can be found in the BI world too.
The same is true in ESOA – and doubly so in the SOA analytics world. If you do not establish a good governance model, pretty soon you will have several services that do the same thing, and hence will beat the purpose of SOA. Unlike in the OLTP world, where things can be traced back to a business object somehow, it is more difficult to solve the problem in the analytics world, especially in a federated environment.
A more compelling argument comes from the service consumer side – like your external business partners who will use your services. If they find several different services that look vaguely similar to each other, you can bet they won’t adopt the SOA world very easily.
5. ESOA approach is not only useful in reading data, it can also be used for writing data into BI
Let us consider the typical BW implementation. You connect your source system to BW, write extractors, transformations etc, and load the data into BW. As time progresses, the number of sources that feed your BW data structures might increase . We go through the ETL set up process again.
Well – you don’t have to do this in all cases. BI lets you write data into info-objects, DSO objects and real time cubes through RFC enabled functions that SAP deliveres out of the box, which by extension means – we can do so by web services too. This can lead us to a very scalable and flexible BI environment.
Once you conceptually design your data definitions, you no longer have to maintain separate source systems, data sources etc every time you have a new system to get data from. Web service becomes the common entry point for data from all approved systems into BI. This is not the solution for all types of data and ETL, and hence you need to use this with some discretion.