Grady Booch provided some interesting perspectives on the future of databases in the context of SOA. Data is actually a critical problem in SOA, we often talk about services that exchange messages that contain "data" as business object representations (REASC: a pattern for constructing Composite Applications), but we rarely talk about data management in a Service Oriented Architecture.
The first major problem is that that all business objects are related with each other (from ERP to CRM to SCM...) and that is not going to change. Relational data models are well aligned with the way humans think, represent, search, access and manipulate data, i.e. they use keys. For instance, some acrostic forms of writings can be thought of as the first indexing mechanism based on letters of the alphabet used as keys (dating several thousand years B.C.), way before the Arab numbering system made it trivial (albeit less poetic) to orient oneself in an arbitrary large document.
Why is it a problem? First services are supposed to be "autonomous" entities, so if you create a "Purchase Order" service and a "Customer Service", they are not supposed to have message exchanges behind the service interface, nor depend on the same database. Second, if your services are truly autonomous, and say you want to retrieve the orders of a customer via some PO service operation, how can you be sure that the PO was recorded with the correct customer "key", how (and where) do you create "views" that join purchase order and customer data? how do you deal with concurrency issues to accept the purchase order if validation rules require attributes from the customer object? ...
Business logic is also a major problem, I would recommend that you read Maarten Mullander's paper "CRUD, Only when you can afford it" (CRUD = Create, Read, Update, Delete) to further understand this issues. I am just reproducing here a small excerpt: "..., most order processing is not CRUD, or at least not according to my definition. For example, an order can be created offline and then sent (replicated if you will) to a service for processing. Processing of that order will affect many of the related entities. The service may update the customer information, potentially changing more than just the year-to-date totals. For instance, the customer might have reached the critical order mass and be upgraded, updating properties used for price and discount calculations; products may or may not be available; delivery dates may or may not have been realistic; and so forth. These changes are important to both parties, but with CRUD, the customer's copy of the order would not reflect them." So it is unlikely that you will be able to cast service interfaces right at the outskirt of the database. To illustrate this point, you might want to read my previous post on "The fundamental problem solved by ESA".
Privacy is another issue: when Data is captured and consumed within the boundaries of an application, we don't have to worry too much about someone "forwarding" it to another consumer, breaking privacy rules that we had agreed ahead of time. In an automated world, it might not be so clear to a particular service that the data it got hold off cannot be sent to another service provider.
Database connections are yet another issue: a service consumer cannot acquire a database "connection", this means that for every message received from a consumer, the provider has to authenticate, authorize and possibly resume the usage of an existing connection or open a new connection altogether.
...
But let's go back to Grady's recommendations
- Database management systems will have to support ACID SOA calls.
- I am not sure what is an "SOA call" onto a DBMS? I assume Grady means that operation invocation will have to exhibit ACID properties, now are these operation invocation directed to the same database? to a federation of databases? Would databases become service providers (and consumers)? In general, out of the ACID properties, Isolation is the hardest to achieve in SOA because resources would often need to be lock for long periods of time, making them inaccessible to other service consumers.
Clearly, we might see in the future that database vendors support the WS-Transaction specification natively.
- Applications will exploit multiple data repositories.
- Yes, these are called composite applications.
- Careful attention to authentication and security will be needed.
- Yes, based on my comment above, this is a relatively complex problem to deal with when related to database connections.
- Distributed two-phase commit will be avoided by recoverable messaging to applications (via services) that consult and modify the database and send a recoverable reply.
- This statement seems to be in contraction with the first statement. 2PC cannot be used in "long-running" scenarios because it relies on resource locking.
- Database size will become a non-issue.
- Based on the cost of hardware this would be true (I have over a terabyte of storage at home (for my PVR) which today costs about $400). We have to be a little more careful though because we really have to relate the size of the database to the performance of accessing and fetching records.
- We'll see lots of low-latency asynchronous replication of reference data among databases serving various applications and their associated service interfaces.
- Yes this is I think the most important and the most valid point that Grady is making, as well as a major difference between composite and monolithic applications. If we go back to our Purchaser Order - Customer example, it is likely that customer data such as address, phone number, contact information, will be stored in the Purchase Order service to avoid costly joins. At that point, replication mechanisms will be needed.
Problems relative to Data in a Service Oriented Architecture are large and complex: they span the conceptual, logical and physical levels of an SOA. We have just begun scratching the surface of the relationship between business object, data, databases and SOA. Issues such as identity, data federation, replication, transaction, privacy, explicit state management,... will all have to be addressed.
The Service Data Object standard will prove to be foundation of not just as a data programming model enabling federated data result sets, but in later versions, as a central vehicle enabling a secure, private, atomic and consistent flow of data in service oriented architectures.
