API Management – old wine in a new bottle. Really !
Internet and blogging sites are bubbling with discussions and debate about API Management and SOA. The so called SOA ‘priests’ claiming an API Management as an extension of SOA while the API ‘purist’ counter claiming SOA’s death attributed to complexity and recommends solution in terms of API Management. In my opinion the debate is non-conclusive and worthless due to inherit concept in API Management that closely follows the architectural principals of SOA while still having the fundamental differences.
How Gartner sees it?
Gartner uses the term “application services governance” as an uber term for defining the SOA governance technology and API management. Gartner, very intelligently keeps the distinction for both SOA Governance and API management while grouping them together under the hood of application services governance indicating the current market trend and possible future consolidation. The market for API Management is increasing, reaching an estimated $90 – $100 million last year with a potential of 40% growth
SOA Governance as a term is going out of fashion and companies are apprehensive to position or brand the products and services in compliance to SOA Governance. However it is still fine to use the word SOA as an architectural principle. SOA as an architecture is not bound to any specific technology although during the peak of SOA hype cycle almost all the vendors has associated SOA with WS-* protocols and that has led to a perception that SOA is implemented using SOAP and WS-* based protocol. On a contrary SOA can be implemented using REST or message oriented middleware.
SOA in the present context shall be treated as an architectural pattern for providing applications functionality in form of reusable services. Service being loosely coupled, self-contained unit of functionality. SOA defines the complete lifecycle of the service starting from identification of business process, designing, modeling, developing, testing, deploying, publishing and discovering along with provisioning, versioning, security, mediation and other aspects of governance.
SOA Adoption and Pitfalls
A decade back, corporates had to interconnect multiple application(s) instances with in or out of his organization. SOA recommended, to expose applications as services and invoking them via sending messages. Web Services, with rigid contracts, specified in WSDL and fixed protocol (SOAP) for message exchange, realized SOA initially. Surely this was much better and standardized approach as compared to B2B ‘type-system’ hell. Thus, most companies embraced SOA principals to manage services (exposing legacy/backend implementations) pre-dominantly within the company and few of them to provide the services to business partner.
In the present context, SOA as an architectural principal can be seen generically- although the ground reality is that it was never emphasized / adopted for exposing services for “omni-channel” consumption, but it was predominantly restricted to be consumed for business/ machine consumption. It is also attributed to a fact that SOA was pulled in the direction (from the technology stack point of view) towards business oriented communication as that was the need of the hour. SOA has always been provider centric as it was more focused toward increasing productivity of provider by enable re-usability and providing service implementation abstraction along with abstraction layer between producers and consumers but offered very little for simplified consumption. Infect, the complexity involved for the consumption of SOA services leads to its downfall.
It was not as much the problem of SOA but more how it was implemented and preached.
In the last 5 years the number of target device for deploying applications (more commonly called as apps) exploded, for example – by an estimate the number of devices by 2020 would be roughly around 10 billion. This leads to the expectation of simplified user experience that now widely varies by the user’s needs, preferred devices, and context. SOA implementation technology proved futile to support such a large spectrum leading experts to find an alternative and giving space to API.
Technically an API (Application Programming Interface) simply defines a strict contract between API consumers and providers – while the process of publishing, promoting and overseeing application programming interfaces (APIs) in a secure and scalable environment is known as API Management. APIs embrace essentially the same goals of SOA but are more open, easily consumable and support human-readable formats like JSON. APIs are also relatively more amenable for external consumption and have strong business affinity in that they can be branded and marketed as products.
Typically an API Management solution has:
Developer Portal: Developer Portal enables the APP developer to discover and explore the published APIs and allows the application developer to register for API access. In short enables the consumption of the APIs for consumers.
API Portal: API Portal enables the API developer to design, develop, configure (manage policy like the security policies for SAML, oAuth, Basic, API Key), deploy and version the API. In short control the complete lifecycle of an API.
Note: – Many of the solutions do not differentiate the Developer Portal and API Portal but provides one portal with different behavior based on the persona.
API Gateway: API Gateway secures, transform and mediates the traffic between your APIs and its consumers based on the configured policies.
Analytics: One of the key differentiate capabilities which allows visualizing usage patterns in multiple dimensions to both APP developer and API developer.
In most of the cases, API Management solution augments the SOA implementations and provides solutions to all the shortcomings of the traditional SOA(P) implementation
Comparing SOA and API Management
Following point access the similarity and differences among API Management and SOA
SOA is typically associated with XML and SOAP while APIs are generally associated with REST/JSON. Tying SOA with XML & SOAP at the implementation level leads to downfall of SOA and a similar care needs to be taken to make sure that APIs are not just tied to REST/JSON. APIs and API Management software shall enable supporting other protocols like MQTT and others which are on the anvil.
Internal vs External
SOA or XML services are pre-dominantly used for exposing services to serve intra organizational use-cases and in some situation for external B2B scenarios as well. On other hand, APIs can be used for both external and internal use-cases although not all services makes a good API thus services should be exposed as an API only after proper thought process and a careful design.
SOA governance to a large extent deal with the design time governance practices. It enables controlling and guiding the development and operation of service through its complete life cycle. The strong emphasis of SOA governance is during designing, developing, testing and publishing of services and weak for runtime execution while the API Management focuses more on runtime governance via policies for Traffic management, Mediation, Security and Extension.
SOA enables Service discoverability using UDDI based registry which lacked human readability and agility. API Management design time governance facilitate easy discoverability and dynamic relationship building typically using tagging and taxonomies.
UDDI based registry failed as it was designed around the concepts of SOAP/WSDL and could not evolve to support protocols like REST & AtomPub. The dynamic binding of runtime service also added the un-wanted service behavior and complexity.
SOA relied heavily on SOAP based services which are more structured and well defined on a contrary APIs are open and well documented which makes them better suited for consumption centric scenario like mass adoption or partner engagement. There is no set standard for API documentation and today multiple options exist like swagger, raml, api blueprint, Apiary etc.
SOA service consumption require web-service client libs and are misfit of the browser/ mobile consumption paradigm. Typically APIs can be self-provisioned with little or no guidance. In my opinion API consumption shall not require additional client side framework or libraries otherwise it becomes one of the major deterrent in simplified consumption.
SOA relied heavily on the security protocols like WS-S, SAML, WS-Trust while API Management offers security via OAuth, HTTP request signing etc.
Top down Vs Bottom up
SOA believed and stressed more on top down approach there by enabling business and service modeling followed by the implementation while API Management till date is mostly bottom up approach which involves exposing the existing services / apis. There are vendors which support and actively enable SOA button up approach as well and similarly the catching trend of API first will also enable top down approach for APIs there by diminishing the difference.
API Management products collect in depth information about API interaction for policy enforcement there by offering a detailed analytics about the API usage and service usage pattern. The insights gathered provide a detailed knowledge which can be applied on multitudes of paradigm right from defining cost structure of APIs to defining maintenance cycle and so on. Traditional SOA offering at large does not provide such insights.
Typically SOA service consumers are less in number and have similar interaction pattern while API consumers by far exceed the number of SOA consumers and behavior of them is also not well known. Similarly the transaction volume on APIs are expected to be more than the SOA services.
SOA focused on architectural approach for organizational & IT transformation there by bringing in indirect revenue (‘A penny saved is a penny earned’). API Management view each APIs as a product and opens up new frontiers for bringing in direct revenue often termed as API Economy. API Economy is forcing corporate to rethink the business model for each API Product requiring separate marketing strategy, forcing KPI based contracts, separate legal bindings along with real time analytics and customer (developer) onboarding.
SOA as was introduced initially and implemented was focused on machine to machine integration and standardization – which lacked the human readability and was inherently complex. SOA as an architectural principal is relevant even today and would remain going forward. Thus it would be fair to say that SOA has shifted its stand from technological piece comprising of standards like SOAP and UDDI to an architectural principle.
On other hand API use the same principle of SOA but are more consumption centric. Due to its inherent consumption centric approach it additionally focuses on developer experience and give rise to new business model.
API Management and SOA shall not be seen as an extreme end of spectrum but more like a complementary technology. The combination of API Management and SOA can help support the use-cases right from Internet of Things up to enterprise back-end systems.