While SOA is generally accepted as a best practice path to agility, the risks for “do it yourself SOA” vs. “COTS SOA” is enormous. And as market hype of “do it yourself SOA” is sun setting in the valley of IT disillusion, the sun of the Process Platform with “COTS SOA” under the hood is rising in the world of business. This blog will look at typical risks in modernizing an application portfolio towards SOA, and the chance of process platform vendors like SAP to position their platform as the marketing hype of SOA stops to resonate with clients.
About the state of SOA, “do it yourself SOA “, and “COTS SOA”
The recent research The Missed Promise of SOA from the 451Group states “SOA has failed to be the IT game-changer it was hyped to be, weighed down by complexity, expensive service engagements, weak standards, lack of skill sets and corporate cultural divides” and “SOA has little relevance beyond the largest organizations.” Prominent critique on SOA started already a year ago, when Cynthia Rettig wrote in her controversial Harvard Business Review paper The trouble with Enterprise Software: “And to the extend that these service-oriented architecture use subsets of code from within ERP and other enterprise systems, they do not escape the mire of complexity built over the past 15 years or so. Rather, they carry it along with them, incorporating code from existing applications into a fancy new remix” SOAs become additional layers of code superimposed on the existing layers.” Joe McKendrick replies to Cynthia’s article in his blog: Don’t expect SOA to fix ERP quagmire: “If not through SOA, then how? SOA, pure and simple, is the first step to software industrialization – creating massive, adaptable systems in an automated and modular fashion through greater economies of scale.”
I think that SOA is an evolutionary and incremental improvement, building on top of a series of improvements we have seen before like: Object Technology, Distributed Objects / CORBA, Component Technology. All these technologies have been hyped as transformational game changers in IT to bring flexibility and reusability to the business, and after the hype has gone, were accepted as best practice in software engineering. However, I also think that “do it yourself SOA” is a very complex endeavor with just too many risks for many organizations. With “Do it Yourself SOA” I mean, that you invest in: a) defining re-usable Enterprise Services leveraging a canonical data model to run the context processes of your industry, b) extending, re-engineering or re-placing your existing legacy applications portfolio to provide and integrate with those Enterprise Services and canonical data models, and c) integrating an application platform that can be leveraged to integrate your and 3ed party COTS packages with these Enterprise Services in order to compose innovative core processes on top of these Enterprise Services (= Composite Applications). On the other hand, I think that “COTS SOA” is a more reliable way forward for many organizations. “Commercial Off the Shelf SOA” means actually quite the same tasks to be performed as in the “Do it yourself SOA” approach, but this time the bold “you” is to be replaced by “a vendor like SAP” and that the approach is more like a normal COTS deployment. Let’s see why…
About Risks in ROI Calculations
Investments and benefits are essential instruments to contrast different IT investment options. However, investments and benefits are not sufficient to forecast the likely return of an investment. A realistic business case must adjust the likely return of an investment with the associated risks. Ignoring risks can lead to very unpleasant surprises as we all have seen in the current financial crisis. The figure below motivates the following discussion.
There are two basic classes of risks: 1) risks that the benefits are not realized (benefit risks), and 2) risks that the investment estimates are exceeded (investment risks). Some common measures for investment risks are comparable references, resource availability (service provider and customer), integration complexity, project seize, estimation template maturity, project methodology maturity (plan/build/run), and technology maturity. Examples for factors that can be used to determine the benefits risks are: availability of productivity references, availability of functional models (references, demonstration, prototype), organizational change management method maturity, performance to schedule, market volatility, cost baseline maturity, operations maturity, as well as software support and quality maturity.
Now, let’s look at some high level common risk themes that differentiate the options.
About Risks in “Do it yourself SOA”
Investment Risks. There is an abundance of Web Services developed by customers today, but only very few of those expose cross industry, re-usable business services based on a canonical data scheme (= Enterprise Services). Therefore, the number of references is very low. The upfront “SOA infrastructure” investment before actually user facing functionality can be addressed, is bigger than with a “COTS SOA” approach, leading to a bigger project seize. Metrics for estimating the necessary “SOA efforts” are usually not as mature like the metrics for a normal project or a “COTS SOA” approach. The integration complexity is huge. Typical complexity levers are: a) re-engineering of the legacy portfolio to expose Enterprise Services without breaking the underlying transaction architecture, b) synchronization of heterogeneous data architectures, c) integrating the application platform to handle the composition and provisioning lifecycle of Enterprise Services. Resources that master the technology toolset for SOA are somewhat scarce, and are available primarily on the consulting market.
Benefit Risks. Current research [Becker et al: „Nutzenpotenziale von Serviceorientierten Architekturen – Ergebnisse einer Expertenbefragung”, Tagungsband des 3. Workshops Bewertungsaspekte Serviceorientierter Architekturen (BSOA 2008), Leinfelden, Deutschland, 2008.] has not yet provided resilient proofs of the usual suspects of SOA benefits, flexibility and reuse. Therefore, the risks of these future options benefits are high. Productivity improvement risks is higher, due to the simple fact that it is basically a new development rather than configuring existing “COTS SOA” processes. Furthermore, end user training has to be developed from scratch, whereas “COTS SOA” providers may already have an existing training curriculum to ease the organizational change management. The complexity of maintaining and operating a “Do it yourself SOA” imposes further risks on the benefits. Last but not least, stating again from Cynthia Rettig’s paper “The timeline itself for this kind of transformation [“COTS SOA”] may just be too long to be realistically sustainable and successful”. This is, I think, a very true statement looking at the speed of the current economic downturn and the fallout for infrastructure projects like “do it yourself SOA”.
Thus, the relative risk profile of “Do it yourself SOA” tends to sit somewhere in the high investment /high benefit risk quadrant, because you have to do a lot yourself.
About Risks in “COTS SOA”
Investment Risks. Usually, there are more relevant references and consulting resources available with COTS. However, most SIs have been struggling to bring together the SAP/Oracle COTS legacy and Java legacy which is necessary to flourish in this new environment. Nevertheless, the reference and resourcing situation is usually better. The upfront “SOA infrastructure” investment before actually user facing functionality can be addressed, is limited to performing a technical upgrade. Assuming that the COTS modules are integrated amongst themselves (= no best of breed), the integration complexity is reduced to the external interfaces. The cost calculation can be based on upgrade metrics, processes metrics and key performance indicators that directly relate to the COTS package.
Benefit Risks. There are usually more references available, where best practice processes from the COTS package have been implemented. This limits the risk exposure of unrealistic user productivity assumptions. Risk not to meet the underpinning functional assumptions, can effectively be mitigated with functional models such as demonstrators, package sandboxes etc. These models help to get the arms around the new system and, along with existing training curriculums, improve the organizational change readiness a lot. Last but not least, the defect rate of COTS is usually lower.
Thus, the relative risk profile of “COTS SOA” tends to sit somewhere in the low investment /low benefit risk quadrant.
While SOA is generally accepted as a best practice path to agility, the risks for “do it yourself SOA” vs. “COTS SOA” is enormous. Why? Because the tasks associated with both approaches are basically the same, but the “COTS SOA” vendors like SAP existing legacy application quality, their existing skills, their research budget, their systematic approach, and their number of trials before getting it [SOA] production ready, is just superior to most user company’s resources. Thus, one should primarily try to leverage the SOA investments of vendors like SAP and expand from there, rather than “doing it yourself”. If you have to go down the “do it yourself SOA” path, than prepare to mitigate the risks. Finally, as the market hype of “do it yourself SOA” is sun setting in the valley of IT disillusion, SAP is well positioned to ride on this disillusion wave and (continue) marketing its Process Platform with “COTS SOA”, aka Enterprise SOA, under the hood… and some customers are interested in what is under the hood and some are not.