Enterprise SOA Explorations: Thoughts on the Effec...

richard_hirsch · ‎02-06-2007

This blog is based on a series of emails I wrote in 2004-2005. The enclosed ideas are still relevant today and I wanted to get them out in the community.

In the ideal Enterprise SOA world, the BPX has the ability to quickly find the services he/she needs in order to build the processes that are necessary. However, as the number of services increases, the supposition that this search is successful or efficient becomes questionable.

I recently saw a SAP Webinar regarding BPM and CAF Capabilities where the emergence of a UDDI 3.0 based search registry was discussed and described as including services that “were classified using semantic rich classification systems”. Although this new registry may provide a technical foundation to support the search, the more important question is: what are these classification systems and what is the origin of the included data.

Caveat: Although this blog will concentrate on searches that involve services the relevant points could be used to describe searches for other elements – for example, Visual Composer models, Guided Procedures elements, floorplans, etc.

At the present time, Enterprise SOA environments are still relatively new. Thus, there are still relatively few services that exist – at least in comparison to the number of other sources of functionality (legacy systems, etc) that are currently available. However, as the environment matures and more SAP back-end systems migrate to ERP 2005, the number of services will multiply rapidly.

At some point in the future, BPXs will be confronted with a huge number of services that are available. Thus, the ability to search efficiently will be of critical importance.

BPX Goals

Before we examine the particulars of the search environment, it is first necessary to take a quick look at the goals that motivate the BPX. I won’t describe all goals – this is impossible inasmuch as these are often project- and organization-specific - but just those goals that in my opinion are related to searches.

The BPX wants to create (or is “develop” or “design“ the correct word) processes in an efficient and timely manner.
Since the development of new functionality is costly, the BPX is interested to re-use existing services and components as much as possible. (Of course, developers of services and components must have the same intention but this discussion is the topic of other blogs.)

Re-use

Based on my own experience, I know that the actual re-use of components – irregardless of whether the environments are SOA-based or not – is often limited. The problems are compounded in larger corporate settings where there may be multiple organizations with different projects and many development teams. Even with a centralized architecture where re-usable components may be available for others, it is often difficult to capitalize on this potential.

By asking a few relevant questions regarding software re-use, we can identify a few critical issues that are relevant for a discussion of Enterprise SOA searches.

Is one reason for the low reuse of components that business analysts can't find existing components? Or is the problem the lack of documentation for the found services?
Is one reason for the low re-use of components that the associated cost of usage is unknown? Project A develops a component and pays for it. What are the usage costs for Project B to be able to use the component?
Or is it a support issue? Why are companies willing to use existing IViews from SAP Business Packages? Because these companies know that there is someone responsible for them if questions arise? Furthermore, the assumption is present that these components have a high quality standard as well as documentation if necessary.

This blog will focus on one important issue that is related to many of the questions posed above: service metadata.

Service Metadata

What is service metadata and what is its relationship to searches for services? Service metadata contains information (quality, technical details, existing users, supported Service Level Agreements (SLA), use in other processes, etc.) for a particular service. In the ideal world, there would be large number of these criteria that would be available for the BPX to use to optimize his/her search. In my opinion, the technical details would be relatively unimportant for the BPX if other more domain-specific criteria were available.

The metadata that would be available is not the same for all services or types of services. Although there may be some elements (usually of a technical nature) that may be standard to all services, there will be other metadata elements that are particular to a certain type of service. Of course, the metadata elements for a particular service type should also be standardized to enable efficient search and comparison. Of course, the Search User Interface (UI) should also reflect these differences and provide the BPX different service type-dependent search criterion (sort of like the SDN Advanced Search)

The source of the metadata about the available services can originate from two sources.

Intrinsic: This information is somehow present in the data. This type of information is usually rather primitive in nature. For example, what services are available, what are their parameters (in/out), what exceptions are possible? The nature of this inherent information is different per information source. For BAPIs, for example, additional documentation regarding domain usage may already be present.
Additive: This information is not present in the information source but must be added through an external source. This information may be created through automatic means (performance measurements from the WAS, etc.) or manually (quality estimates from external consultants).

The collection of metadata

Of course, this metadata has to be collected somehow. There are different avenues to achieve this collection. Examples are:

The SOA provider itself.
Users of the SOA (rating system).
Mechanical means. This refers to information/metrics that can be collected automatically. For example, how often has the service been used, how often has it crashed, how is its performance.
Through external agencies. I can imagine that there is a huge market for consultants who are hired to examine a company’ existing services and create the desired documentation that is necessary to optimize searches for services. A company might have many services that just have technical (if that) documentation/information. An external consultant could come and conduct interviews and analyze code to gain the information that would be important for the BPX. These consultants could base their data collection on the service-type-specific information standards mentioned above.

Of course, the more metadata that a service can provide, the more useful it becomes for a BPX; thus, one search criterion could be to what percentage does a particular service meet the metadata standards. For example, a BPX might search for services that have a minimum of 80% of the necessary metadata necessary to meet a particular standard. Thus, the services with a higher quality standard would be selected more often which would lead to an overall quality improvement of the process landscape.

Quality of information

"Quality" could also be a filter that is present when searching for information. "Quality" in this case refers to the extent that data meets the information standard established by SAP or others. If I can choose between a service that is well documented, is hosted by a well known company and has OK performance and a service that meets the same domain requirements with no additional / additive information, I would rather use the first service, because I know what I am getting when I use it.

There might also be a new niche market for consultants who come to a company to measure the quality of their offered services. These consultants would then provide an objective seal of approval (“This service has been examined by Acme Quality Service Consultants and found to be of Prime Quality”) that could be used in the search.

There could also be some sort of new ISO Quality Metric based on the number of services that a company offers combined with service-specific quality characteristics.

Relevance to ESR

The Enterprise Service Repository (ESR) plays a critical role in SAP’s Enterprise SOA environment. Although the success of the ESR is partially based on its technical characteristics (high performance, high availability, etc), far more important is the ability of users (in this case, BPXs) to find the services they need. The ESR can host hundreds of thousands of services but if the BPX can’t find that particular service that he needs in a rapid and efficient manner, then BPXs will probably be unsatisfied with the tool. Thus, the use and availability of service metadata as described in this blog is critical. Its inclusion will improve the efficiency of BPXs and thus increase the efficiency of the processes on which they work – and this goal must be at the heart of any company’s BPM effort.