I knew that one major advantage of Enterprise SOA / Composite Applications was the more efficient usage of legacy applications. After I had some understanding of the SAP technology available for this purpose, I started to examine the existing architectures of various applications that I knew existed in our organization. Many of these applications were based on databases. Thus, I thought it would be useful to develop an understanding of what possibilities were available to integrate these applications into an Enterprise SOA landscape.
This blog is not meant to be dogma – the last and final truth about database integration in an Enterprise SOA world – it should be seen as a starting point for discussions in this area. If someone disagrees with my arguments, then the resulting corrections will help clarify options available for those facing similar decisions. For as we all know, the scenario described is quite prevalent and appears in numerous projects.
In this blog, I won’t focus on the databases of a particular company or applications of a particular type. Rather I wish to examine at an architectural level, what options are present and attempt to create guidelines regarding which tool is appropriate for which usage scenario. I am not seeking a magic wand that solves all problems for all scenarios but guidance when making decisions.
This attempt is not as easy as it might appear inasmuch as the scenarios are affected by various factors that reflect application characteristics including:
- Basic Usage scenario: Is the application to be created for learning purposes / a prototype or is it an application with a demanding SLA – 100 % availability. Is the application going to be accessed by 10 users or 1000 users?
- Legacy application or new application: If the application is a new application then I have some freedom regarding the database and its characteristics. If the application is a legacy application, then my possibilities to change the application maybe limited.
I will examine a general scenario involving the integration of a legacy database to explore this issue in detail.
Legacy Scenario Assumptions
I am assuming that I am creating an application that is being constructed using the SAP technology used for Composite Applications (including Visual Composer, CAF Core and Guided Procedures). For the sake of completeness, let us also assume that there is a business process model in ARIS that describes the steps that are necessary.
Those roles involved in creating the application are the business process expert (BPX), the technical architect and the developer. The BPX has experience in Visual Composer / GP but does not have deep technical knowledge.
The database to be integrated is a legacy application. The goal is to allow the BPX to access this database via Visual Composer and from Guided Procedures. The BPX should be able to work in a fashion that meets his needs and does not place any expectations on his know-how or capabilities that are not based on his role in an IT/Enterprise SOA environment.
What are our options to achieve the goals in our scenario?
Direct access: One of the easiest options to access the legacy database would be to give the BPX direct access to the data in Visual Composer using the tool’s ability to tap into JDBC data sources. There are various tutorials that describe the process involved. Once the JDBC data source has been made available to VC, there are two possibilities – either through the direct usage of “Select” queries using Business Intelligence (BI)-related or indirectly through the use of stored procedures.
The problems associated with this option are:
- Many BPXs are not able (and actually shouldnt be expected) to know SQL syntax. As we all know, the required SELECT queries are often quite complicated and require detailed knowledge of the appropriate database schema (table, indexes, etc) as well as detailed knowledge about SQL syntax (joins, etc).
- In the ideal Enterprise SOA world, the BPX should not have to know that the data he requires is based in database. He should have his selection of services that meet his domain requirements. The origin as well as technical characteristics of this information are largely irrelevant.
- This solution is just appropriate for Visual Composer. In the Guided Procedures environment, there is no possibility to directly access databases.
Indirect access:. This option is based on the necessity of wrapping the database in some sort of a web-service. I will examine the technical implementation options for this access later. This option has the benefits that the BPX can access the Building a Web Services Based Visual Composer Application or through Guided Procedures (via the Web-Service callable object)
My recommendation is for the BPX to use indirect access (via a web-service) to access the legacy database inasmuch as this supports the Enterprise SOA model and allows the BPX to use an abstraction construct that is more appropriate for his technical know-how.
Now lets examine what are our options for implementing this access.
Implementation Options for Indirect Access
Implementation Options for Indirect Access: External options
The first set of options is associated with technology that exists outside the SAP Web Application Server. For example, the legacy database may be part of a legacy application. If possible, this legacy application could be enhanced to provide web-service access. Of course, this requires that the legacy application have the technical foundation to be accessed via HTTP/HTTPS. Another option is to use the tools associated with the database itself. Many database providers provide the ability to access their database products via web-services. Unfortunately, this option often requires additional development work in the database server and/or often requires an additional Application Server.
If a legacy application does not exist or can not support web-services, then it may be possible to develop a small wrapper application that runs with a small footprint (for example, Tomcat with AXIS) and accessesthe database via JDBC and provides web-service access to the data.
The problems with these options are that they often complicate the server landscape by increasing the number of servers that must be operated and maintained. Of course, one could take a long-term approach and use a web-service-enabled legacy application as an initial point and migrate it to more Enterprise SOA relevant environment at a later stage.
Implementation Options for Indirect Access: Internal options
These are options that are based on the Web Application Server. If technically possible, these options should be promoted due to performance reasons as well as operational simplicity.
You might think that CAF Core might be an option here but it is not. Due to the fact that it is not possible to create Entity Services based on existing databases, it is not possible to use this option. Of course, when database-related web-services exist, then it is possible to integrate them into CAF-Core as External Services.
What are other options? Web Dynpro is one option to access our legacy database and there are various possibilities to access databases in this framework and then present it as a web-service. It must also be said that that Web Dynpro is more difficult to learn than CAF Core; therefore, it has more capabilities than CAF Core. The tools, although in many ways similar, are used for different reasons. You might see the two as lightweight (CAF Core) and heavyweight (Web Dynpro) approaches to solve comparable problems.
The question is always when do I use which tool? This question is unfortunately not easy to answer and is dependent on a variety of factors. But in general, I think CAF Core is better when you already have existing services that you want to enhance. Web Dynpro may be better when you create web services from scratch. The fact that many of the CAF Core API’s are not currently available in the SAP JavaDoc may be another indication of this tendency.
The last option is the development of a J2EE application with EJBs that is deployed in the Web Application Server. Of course, this option permits the usage of all J2EE capabilities regarding database integration but brings with it a certain complexity that increases development time, etc.
A Note on Non-legacy Database Access
With CAF Core, it is possible and very easy to create Entity services that are based on local persistency which is appropriate when creating applications based on a database where the database does not already exist (non-legacy). This means that the database-related activities themselves are taken care of by the framework itself. CAF core stores all the created data in the WAS database. You don’t have to create tables or indexes; CRUD operations are generated automatically. If you look at the logs during deployment, you can see all the database actions that are being performed. This makes the job of the developer easier in that most database-related activities are hidden and the developer can concentrate on the business logic.
This usage is ideal where educational purposes or prototypes are involved. Turn-around is very rapid and one can concentrate on other matters.
The disadvantages are actually related to the above mentioned advantages. All these database-related activities take place without your control. For example, if you wish to place your application data in a different database and still use local persistence then this is not possible. Of course, you can wrap your external database (as I discussed previously) as web-services but then you lose the ease-of-use advantages of the local persistence based entity services.
Furthermore, I have certain reservations about placing the application data in the same database as data from the application server itself. An application that produces heavy DB load would influence the entire WAS and all applications running on the WAS.
One suggestion that was made in a post in the “Developing Composite”Forum was to have a separate WAS that is dedicated to a single application. In a productive environment, I have reservations regarding this solution based on cost reasons – you have to maintain and operate a WAS just for one application. If you have multiple Composite Applications using the WAS database as their data source, then these applications must have the same SLA and similar user bases. If one application has 1000 users and has availability from 24/7 and a different application has a SLA with 90 % availability and 100 users, then their hardware and operations-related requirements are quite different. Combining these two applications in one database may be troublesome. Furthermore, the ability to use normal database tools to load or effect data would also more difficult.
Based on the requirements of our initial scenario (the integration of legacy database that is not already wrapped), I think that Web Dynpro presents the better option. I would love to see CAF Core be able to do something like this but at the current time, it is not possible.