A beginner’s perspective on ESOA: Part I- Evolution
ESOA: An architecture which is services-oriented; It is difficult for a beginner to understand its purpose and the fact that it is important for SAP. So, let’s just assume that this ‘service-oriented architecture’ is meaningful and then try to find out the reasons or requirements which led to its evolution. We will discuss the terms and concepts of SOA in part II of this blog.
Here I will try to find the challenges and requirement (both business and IT) which are responsible for the evolution of SOA from its predecessors.
To start with, let’s look into the history of ERP. Before the concept of a huge application that plans for the entire enterprise, i.e. ERP, organizations used Best of Breed architecture. That is to say they relied on different systems focussing on different domain (or business processes) of the enterprise. For example different systems were used for Finance and accounting, HR, Sales and so on. Typically all these systems were coming from different vendors and they knew nothing or very little about the other systems .And obviously they had different databases and more importantly different data models (for example a system might model a ‘material’ differently than others). So to use these different systems within the enterprise it was required that they communicate with each other. By communication I mean the data exchange between these systems. Having said that they don’t share a common data model, middlewares and interfaces were required to establish the communication (data exchange). So the entire enterprise would look like this:
This architecture is plagued with what is known as N (N-1) Integration problem. This means that we require N (N-1) interfaces (shown by arrow heads) to integrate N systems. Moreover if any new system is introduced then we require another N interfaces so that this system can communicate and exchange data with all the other systems. (Note the use of term System and not Component)
The evil behind this problem was that all these systems weren’t sharing a common data model.
And due to this reason it was very difficult to have real time data change/updates within the entire enterprise. All the systems needed to be synchronized which was not possible because there was No Coupling between these systems. So the process goes something like, Sales & Distribution take orders and store them in a database or in the file system locally, and at the end of the day batch process all these orders and communicate with other systems, for example, check the inventory or send production order or production plan.
To summarize we have two major difficulties with this architecture:
- Creating and maintaining interfaces
- No real time data updates
After realizing the importance of having a common data model for all business processes the concept of ERP evolved which said “let’s plan for the entire enterprise”.
In that regard SAP released SAP R/2, based on mainframe technology. Now there was one application for the entire enterprise and therefore all the components corresponding to business processes shared the same data model, in fact same database. So we moved from No Coupling to very Tight Coupling. Also now, since all components understand each others’ data, they can work together; which resulted in real time data updates in the system. That is to say it was possible now to update inventory, or a production order upon receiving a sales order instantly. So the use of common data model in SAP R/2 solved the two major difficulties with its predecessor.
The ‘R’ in SAP R/2 and R/3 stands for Real-Time.
SAP R/3 used the same approach but moved to Client/Server architecture for the sake of scalability. However, unification of all of enterprise data in one data model was considered rigid as new business requirements grew. If you change the type of material number from char 30 to char 45 in the system then it is reflected everywhere, whether it is required or not. This prompted to (somewhat) decouple all the components by the means of metadata. The system now has metadata as well, to define for example, how the same employee data would look in Finance and in HR and so on. The concept of multiple clients was introduced and these interacted with each other using RFCs. Moreover components of ERP started to communicate and exchange data with each other using technologies like ALE/EDI.
From the above diagram this conclusion should not be drawn that all these components are decoupled, in fact they were still dependent on R/3 and were tightly coupled to each other. But it could be said that we moved from a very tight coupling to a looser coupling.
As ERP established, vendors started to build systems that would sit on an ERP and provide additional functionalities, like CRM. Although these systems were developed on different platform but they were developed keeping in mind an underlying ERP system and they provided custom interfaces to communicate with the ERPs of big vendors(like SAP). Even SAP started creating such systems, like CRM, SCM, SRM.
To incorporate integration of such systems with the existing ERP model, concept of Business Objects Repository was introduced. Now the enterprise data was modelled in the form of business objects (and the collection of these objects was called BOR, Business Object Repository) and only “methods” of the corresponding object can modify the data contained in the BO. These methods were called by using special kind of function modules which were RFC enabled and were called BAPI (Business Application Programming Interface) Function Modules. These had well-defined and (remotely) invokable interfaces and they were used to exchange data to and from an ERP system. This Concept enabled integration of different systems (of different vendors, implemented on different platforms).Prior to BAPI other technologies like ALE and EDI were used within SAP to integrate different applications. What BAPI provided was a way to integrate external applications into SAP ERP. So now the architecture would look like this:
The question which were trying to answer was why we need a services-oriented architecture ( and what is it anyway) ? With the present architecture the following issues(scenarios) may arise:
a) Consider a business process that involves multiple applications like ERP, CRM and SRM. For example a order-to-cash process involves multiple applications – customer-facing applications, such as customer relationship management (CRM), supplier-facing applications, such as supply chain management (SCM), and enterprise resource planning (ERP) systems where the order resides, and where all transactions and entities are stored. In a typical scenario, middlewares and plugins(which internally use BAPI calls) are used for integration however human intervention is not completely eliminated in the integration process, or , in even worse scenario employees act as human integrators, sitting in front of many different applications, transferring information from one to the next by copying and pasting and retyping information, making process-flow decisions as needed. The applications are hardwired through brittle connections that are (possible but ) expensive to maintain
Read Enterprise Services Architecture-Introduction for more details of this scenario. The problem is somewhat similar to the integration problem we faced earlier only now the systems interacting are much bigger in scope and purpose.
b) Consider a scenario where an organization has acquired an ERP system from vendor V1, however some of the components (lets say HR) do not map to their business process (this is a hypothetical situation, in a more practical scenario they might want some changes/enhancements in the component) or they realize that there are other vendors who offer HR solutions that are better suited for their organization.
So now they have an option of asking V1 for enhancements in that component (this is what happens until now) or somehow try to integrate the HR solution from vendor V2 into the ERP system of V1. If the organization decides to go for the later choice then how can they accomplish this while still using their existing ERP system.
c) One final problem is of redundant and non-reusable programming. Consider a bank that has several (well integrated) application systems (like ATM system, credit card scoring system etc) developed for a different line of business within the bank and at different times. Thus, for example, the function of get account balance is repeated in the ATM system, the branch teller delivery system, and the credit card scoring system, even if they access the same account data in the same database. This is because the get account balance service is implemented keeping in mind its application and tightly coupled to it. Similar situations can occur in ERP or any complex software application for that matter.
A services-oriented architecture answers all these problems, but before looking into that lets just summarize the requirements:
1) Complexity of Integration:
Leverage existing assets while upgrading the architecture or landscape
Coping with changes within an organization (mergers, acquisitions, new subsidiaries)
The ability to communicate with other applications using standards-based mechanisms and simplifies connectivity across heterogeneous landscapes.
2) Redundancy in Programming:
Avoid implementation of a service repeatedly due to the fact that development is happening at different time with different business purpose and for different applications.
3) Reduce Interfaces:
Bypass the N(N-1) interfaces problem or use some intelligent techniques to create and maintain several interfaces within one integration project.
- Migrate to Service-oriented architecture part-1: http://www-128.ibm.com/developerworks/webservices/library/ws-migratesoa/
- Migrate to Service-oriented architecture part-2: http://www-128.ibm.com/developerworks/webservices/library/ws-migratesoa2/
- Enterprise Services Architecture : http://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/media/uuid/e084339e-0701-0010-bea7-e5d5e8ab5f09
Next blog in this series A beginner’s perspective on ESOA: Part II- Architecture