Microservice Integration Patterns
If you replace a monolith with a distributed Microservices architecture, you need a sound integration strategy. Based on the practical experience with the SAP Community, which is based on a sophisticated Microservices architecture, this blog post presents you fundamental integration patterns that can be categorized into integration with UI, messaging and APIs:
- UI: The integration on UI level is very important because it helps to minimize the number of API calls or messages. If you successfully avoid building a monolithic UI, then you need to somehow integrate multiple smaller UIs into one connected user experience.
- Messaging: This powerful technique connects systems without strong coupling. While synchronous communication requires the client to wait for a reply, asynchronous communication does not require message producer and message consumers to work simultaneously.
- API: This kind of point-to-point integration with Remote Procedure Calls introduces the strongest coupling between the connected Microservices. The communication is always synchronous because HTTP follows a simple request-response pattern. This is probably the most common approach to Microservice integration but it should be avoided whenever possible with UI or messaging integration.
Integration focuses often too much on APIs and the advantages of UI integration are overlooked. What are the advantages?
- Avoid monolithic UI: Microservices should provide their features to users with their own user interfaces. In contrast, a monolithic user interface would eliminate the modularity advantages of Microservices because, without much doubt, the monolithic user interface would be the bottleneck of scaling agile teams.
- Minimal coupling: You could, for instance, use a simple link to navigate from one system to the other.
- Time to production: If frontend and backend are developed by the same team, they can adapt both ends more quickly. For example in Scrum, a team should be cross-functional, i.e. the team can develop a working product from start to finish. If the team would be limited to a technical layer, it could never deliver continuously.
UI Integration with Hyperlinks
Simple links can be used to navigate from one system to another:
UI Integration with Transclusion
Transclusion is a very common technique in the web. HTML relies heavily on the transclusion of images, scripts or stylesheets. There are many technical options for transclusion:
- Edge Side Includes (ESI) is a simple markup language and can be used for used for server-side transclusion. It defines web page components for dynamic assembly and delivery at the edges of the Internet. ESI tags are usually inserted into HTML files and then processed by Content Delivery Networks such as Akamai.
- Server Side Includes (SSI) are quite similar. This server-side scripting language is usually used to include the contents of one or more files into a web page on a web server.
- Ajax can be used to implement transclusion on client-side.
Where is transclusion used in the SAP Community? Let’s take for example the web pages implemented with the content management system Adobe Experience Manager (AEM). The web pages have static and dynamic content that is included either with Ajax or SSI. The SAP Community uses Akamai but decided to not use ESI to keep the architecture as simple as possible.
SSI is used in this context for dynamic content that can be cached for a determined time interval together with the static part of the web pages. The dynamic content is included in the static page before it leaves the Dispatcher (Apache HTTP server) and both get a single Cache-Control HTTP header. This technique is used for user-agnostic data that can be cached by the Dispatcher and by Akamai.
Ajax is used for dynamic content that is cached for shorter time intervals and for content that is not cached at all. In this context, user-agnostic data is loaded with Ajax from AEM that is connected to upstream Microservices. User-specific data is loaded with cross-origin Ajax calls directly from the Microservices without help from AEM. The SAP Community uses cross-origin calls if caching is not possible.
It is important to notice that the connected Microservices provide dynamic UI components that are transcluded into the static pages from AEM.
Messaging has a lost of advantages in comparison to synchronous API calls.
- Publish-subscribe: The SAP Community uses a topic-based publish-subscribe model that ensures that the message producers do not need to know the consumers. The number of producers and consumers is variable.
- Asynchronous communication: The message producers are not blocked by the message consumers. This improves the availability of the systems since the messaging system buffers the messages until the consumer is able to process them.
- Guaranteed delivery: Messages are not lost if a consumer is unavailable if the messages are stored by the messaging system.
- Extensible: New Microservices can be connected to the messaging system without architectural changes.
- No service discovery: The Microservices send the messages the messaging system.
The most common approach to connect Microservices is the usage of APIs. I mention them at the end of this blog post because UI integration and messaging are often overlooked. Of course, you can build your UI on top of your backend APIs. These APIs are not published, they are only used “internally” and both ends are under the control of one team. Before you, however, publish an API to external clients, you should check UI integration and message-based integration.
Design Best Practices
- Design your API with resources and apply a consistent URI convention. Do not use a singular service endpoint that processes all different kinds of actions.
- Obey the semantics defined in RFC 7231 when you chose the HTTP verbs and status codes.
- Use hypermedia controls.
- Make backward-compatible changes whenever possible and try to avoid introducing new versions.
Anti-Pattern: Database integration
In theory, multiple Microservices could use one single centralized data model and one centralized database. This integration approach is fairly easy and simple but introduces a tight coupling between these systems. Therefore, this anti-pattern should be avoided. There are exceptions to this rule if you treat the connected systems as one.
There are basically three options to integrate Microservices. The easiest and most familiar approach is the usage of APIs for Remote Produce Calls. You should however not overlook messaging integration and UI integration, which offer many advantages.