Microsoft BizTalk 2009: A deadly competition to SAP PI, fast and flexible, armed with all kinds of adapters, including proprietary protocols.
Performant and unbelievably adaptive as it is, still, MS BizTalk remains a messaging-based middleware. This is no problem as long as You want to employ BizTalk to exchange messages. Without dwelling too much on details, most kinds of communication models used nowadays in enterprise environments, including SOAP Web Services, fall into this category. But what if… What if You want to implement a Resource Oriented Architecture (ROA) kind of a solution?
What is this ROA thing at all then? The term itself, coined probably in the book “RESTful Web Services” published 2007, denotes integration architecture based upon RESTful principles as described in chapter 5 of Roy Fielding’s doctoral dissertation Architectural Styles and the Design of Network-based Software Architectures. Until recently ROA has been a phrase to be heard among pioneer integration developers, mostly in the world of Microsoft WCF and on MSDN. In order to get the feeling how new and unpopular this term was, it is enough to visit Wikipedia article devoted to that topic and take a brief look at the comments on top:
Nonetheless, it is enough to run a quick query to find out that ROA has been spreading like bushfire. Inherent simplicity and robustness of the ROA model appear to gain popularity in the cross-brand integration world. This blog, however, is not a place for a discussion why and to what an extent RESTful web services are often more suitable than SOAP-based ones. What I would like to concentrate upon is a case study on ROA implemented with Microsoft BTS 2009 in SAP landscape.
Having briefly explained basic terms, let’s come to the point.
This is a real world scenario implemented at a multinational corporation from the Oil & Gas sector. Heterogeneous system landscape consisting of various specialised in-house developed systems and SAP IS for Oil & Gas was supposed to be integrated based upon BizTalk server grid. Due to administrative reasons, each country-specific affiliate should govern its own BizTalk server instance. This is where entire interface-specific mapping from proprietary to canonic message format is conducted. In order to ensure proper message routing, a central BizTalk server instance is established. Central BTS instance is not allowed to interfere with message contents but merely routes it to an appropriate recipient.
To briefly sum up:
- Business logic is enclosed entirely in local BTS instances.
- Routing logic is performed exclusively in the central BTS instance
- There is no separate service directory. Central BTS server identifies request recipients and performs the routing based upon routing rules stored in Microsoft Business Rules Engine
Skipping pros and cons analysis as regards solution options, a ROA implementation in this scenario has several advantages. E.g., it is only necessary to have just one single routing orchestration in the central BTS instance where routing is done based solely on message-independent request properties like URI or HTTP header values (method, MIME type, Accept and so on). Provided that we can capture those request properties (I will come to this later on), we can easily establish rule-based routing scheme. In this scenario, Microsoft BRE (Business Rules Engne) in conjunction with an MS SQL routing configuration table has been used for that purpose. Not without some tuning though…
Can BizTalk do ROA?
Not exactly. As said, MS BizTalk is primarily a messaging-based middleware and RESTful communication does not necessarily require a message (e.g. HTTP GET request cannot have body at all). Needless to say, there is no built-in support for RESTful kind of communication either.
On the other hand, since version 2006 R2 MS BizTalk has been happily living in a marriage with WCF framework in form of a set of adapters. And WCF framework has been succesfully used to implement RESTful scenarios, in fact, since its introduction in .NET 3.0. However, there is no ready to go soultion for that.
Can NetWeaver do ROA?
No, not really. Nonetheless, thanks to the flexibility of the NetWeaver Internet Comunication Framework, it is not that difficult to “teach” NW speak RESTful language. In 2004 DJ Adams wrote an excelent introductory article showing how easy, in fact, it is. Some time ago I used the same principles in order to prove that SAP PI can do RESTful web services just as well.
The bottom line is, however, that development effort is reuqired in order to use RESTful web services in WAS ABAP.
Right… So why ROA???
With consideration of the fact that neither Microsoft BizTalk nor SAP NetWeaver support RESTful web services out of the box, this question appears sound, by all means. So how come that a customer has chosen this solution?
Scalable message routing
RESTful web service addresses resources. Every resource has a unique URL. Hence, contrary to SOAP Web Services, addressing is not part of a message. This seamingly simple statement has some important consequences. Namely, a recipient of a request may, without interfering with the message’s contents or without even message being present, understand what object is being addressed and act accordingly – e.g., re-route the meesage to another recipient. All that is needed is a set of rules, based upon the URL. Moreover, all kind of requests are routed the same way and over unlimited number of intermediaries. Complete separation of request routing from message itself makes this mechanism incredibly robust.
Native support for binary messages
For several reasons, transporting binary data over SOAP Web Services is not particularly easy. There are various standards for this otherwise simple activity, standards which obviously collide.
On the other hand, RESTful web services using HTTP as application protocol do not require anything else than MIME multipart standard dating back to early nineties. A multipart message can transport any number of parts comprising any kinds of data. Such a feature may come in handy as regards e.g. sending document data in textual form (e.g. as XML) and its scanned picture as a binary file (e.g. PNG), all in the same message, nice and tidy.
Simple and effective security
RESTful web services using HTTP as application protocol may be easily secured with TLS (SSL) which can not only provide transport security but also client authentication. This kind of security is used, among others, in the internet banking, shopping and all other kinds of internet-based activities and – more importantly – so far it has not been compromised. Obviously, WS-Security may still be used but… Why should it?
Another layer of security, specific to the RESTful web services is associated with resource-based addressing. A request may be allowed or denied (audited, reported etc.) based upon the very URL and HTTP method in conjunction with client information. E.g., client A may be allowed reading of a resource XYZ but denied everything else. All without even touching the message belonging to the request!
Elegance may not be the key word to have the customer’s ear. On the other hand, efficiency may very well be. With a very effective communication model, simple but brilliant security and unmatched flexibility, RESTful communication provides for an elegantly efficient solution that may excel over its direct competition – SOAP Web Services.
So what’s the story line?
SAP WAS ABAP
The logic here is rather simple. For a client scenario, data are extracted using business interfaces (BAPIs), converted to XML using ID transformation, stripped off asxml envelope and sent over HTTP to the local BTS instance. The only interesting bit here is the URI and HTTP method, all dependant upon actual business object processed and action.
For a server scenario, request is analysed in a custom ICF handler implementation and – depending upon request URI suffix and HTTP method – relevant business object is identified. Afterwards, XML data (HTTP message body) is enriched with asxml envelope and converted into ABAP. Final step covers calling business interface with the extracted data.
BizTalk Server Enhancements
As said, BTS is not natively capable of RESTful communication. Hence, several enhancements on both Send Port and Receive Location are needed.
As regards BTS Send Port, it is already possible to set send URI dynamically. However, HTTP WCF Adapter for Send Port must be made capable of passing HTTP method. This can be achieved using WCF Extension (IMessageInterceptor). Also, in case of outgoing requests which are not supposed to have any HTTP body (like GET or DELETE) another extensions must take care of that,
BTS Receive Location is a very different story.
Existing documentation for WCF adapters covers mostly using isolated instance case (WCF service hosted in IIS). However, it is not necessarily required as the very same adapter may also be used as a standalone HTTP host. There is a catch though – WCF Web Programming Model from .NET 3.5 cannot really be utilised as we can only extend BizTalk WCF Adapter code. Hence, old fashioned .NET 3.0 way for implementing RESTful services must be used. Keeping this in mind, everything goes just fine. The following modifications are necessary in order to enable RESTful communication for WCF Receive Location:
- AddressFilter property of the endpoint dispatcher must be modified so that any URI (or prefix-based) is accepted
- Request URI suffix and arbitrary HTTP header fields must be captured and passed on to the BTS orchestration
- For HTTP requests without message by default (like GET), a dummy message must be constructed so that BTS can process it
The last bit is the orchestration itself which should be able to understand and interpret RESTful message properties. The solution is storing those properties as custom message context attributes.
Generic Message Routing
Once all the important bits of the request properties are available in a message context, Microsoft Business Rules Engine can do the dirty job with the determination of the message recipient. Using BRE provides for an additional abstraction layer, hiding the implementation of the routing rules from the orchestration. In this solution routing rules have been stored in a MS SQL table but thanks to BRE flexibility the rules can be retrieved from just about anywhere.
- Routing information is separated from the message itself, hence BTS instances can easily be chained
- Flexible routing based upon requested response representation type (HTTP Accept header field) is possible
- Multipart messages can efficiently be used, also with parts coming from / sent to different routing targets