B2B Digital Ecosystem – Event-Driven Architecture (EDA) – CPQ, BTP Event Mesh, SAP CPI, Microsoft Azure
B2B Digital Ecosystem Connect applications, services, and systems across different landscapes.
Recently i have got a chance to contribute to Design & Implementation of Eco System using Combination of Microservices &Event-Driven Architecture (EDA). In this blog post I would like to demonstrate how to build Eco system by
- Integrating BTP Event Mesh in CPQ
- Event Mesh Integration in SAP CI (aka HCI) iflow using the AMQP Adapter
- Event Mesh Integration in NodeJS or Java
- Event Mesh Integration in SAP BTP Applications like CAP
- Event Mesh Integration in 3rd Party systems with Microsoft Azureto send Emails, Calendar Events.
Basically, all these systems, applications, services across different landscape participate in event-driven business processes across business ecosystem for greater agility and scalability.
- For smaller projects with a simple application requirement, monolithic may be a right choice. Many forces affect software in the past few decades: larger datasets, geographical disparities, distributed complex company IT structures, complex tasks. Increasing complexity in infrastructure, application, and data size requires new architecture approaches. As you know In a monolithic application, the core problem is this: scaling monolithic is difficult. The resultant application ends up having a very large code base and poses challenges regarding maintainability, deployment, modifications, scalability reliability and availability and agility. This is because Tight coupling between components, as everything is in one application. Microservices promise to help break down monolithic applications and enable the consistent delivery of services.
- New trends such as Artificial Intelligence , IoT , RPA [SAP Intelligent Robotic Process Automation ] are redefining what is possible in many industries. Originations has to achieve a Intelligent Enterprise Eco System that can be flexible, adapt to all changes ,trends, and make decisions in real time.
- Organizations must adopt Enterprise Messaging event-driven architecture to their systems and applications to improve the agility, scalability and responsiveness of applications and access to the data and context needed for better business decisions.
- There are plenty of articles online which describe the significance of an Event-Driven Architecture for enterprises today.
- Event-Driven architecture, a software architecture pattern that promotes the production, detection, consumption of and reaction to events.
- Asynchronous messaging is a type of loosely coupled distributed communication that is becoming increasingly popular for implementing event-driven architectures. Event-driven architectures are ideal for improving agility .
- If you would like to Create responsive applications that work independently and participate in event-driven business processes across your business ecosystem for greater agility and scalability then Intelligent enterprises use SAP Event Mesh to trigger process integration with one or more related applications. With the emergence offerings like SAP BTP Cloud Foundry, the developer community had everything it needed to implement scalable business application.
- As per definition an event mesh is a configurable and dynamic infrastructure layer for distributing events among decoupled applications, cloud services and devices
- SAP Event Mesh (aka SAP Enterprise Messaging) running on SAP Business Technology Platform (SAP BTP, aka SAP Cloud Platform) is a fully managed cloud service to connect applications, services, and systems so they can interact with each other through messages and asynchronous events .
- Allow events from one application to be dynamically routed & received by other application no matter where these applications are deployed :On Premise, Private Cloud , Public Cloud , IoT Device ,Mobile Device , Any moving object like any vehicle
- SAP Event Mesh, a cloud-based capability, allows applications to communicate through asynchronous events, and extend your digital core.
- Asynchronous & Decoupled communication is magic here.
- High availability clusters and elastic scalability is capability of EMS
- With EDS surely performance can be improved as you will plan to move time taking tasks to EDA & Microservices
- SAP Event Mesh supports SAP’sstrategic event-driven architecture
- SAP Event Mesh is available on SAP BTP Cloud Foundry as part of the SAP Integration Suite.
- Enterprise Messaging is a native SAP service which out of the box allows one to consume events from your business systems such as S/4HANA, and SFSF I,e without any additional setup, development, you can configure your S/4HANA system/tenant to publish events to the BTP
- Intelligent enterprises use SAP Event Mesh to trigger process integration with one or more related applications.
- An event-driven architecture consists of event producers that generate a stream of events, and event consumers that listen for the events.
- Event-driven architectures have three key components: event producers, event channel, and event consumers. A producer publishes an event to the router, which filters and pushes the events to consumers.
- A Message client is the central component to define the access from / to Event Mesh, It also controls the access to topics and queues
Please check for more material the following sources:
Also Follow this blog by @PRADEEP PANDA to get complete details about Event Mesh Concepts
An BTP event mesh can support a wide range of use cases like E-Commerce ,Sales (Example : Automate end to end sales process) , Finance (real-time trading), IoT, support fraud detection kind of usecases
While point-to-point communication is fine for sharing data between a limited number of senders and receivers, scalability can quickly become an issue. SAP Event Mesh solves this issue and ensures messages can be exchanged reliably between senders and multiple receivers at large scale. SAP Event Mesh is a powerful cloud message broker and event-enabling service.
1) Real Time communication : Events are notified in real time so that consumer can respond immediately to events . The idea of EDAs and asynchronous communication between components is to have close to real-time communication between these components
2) Decoupled communication: Producers are decoupled from consumers — a producer doesn’t know which consumers are listening. Consumers are also decoupled from each other, and every consumer sees all of the events. you can use an event-driven architecture to share information between systems without coupling i.e The BTP EMS removes the need for heavy coordination between producer and consumer services, speeding up your development process .Asynchronous & Decoupled communication is magic here.
3) Support Performance improvements by Non-Blocking: Sender doesn’t need to ask the consumer all the time if there are processing resources available, or if some task is completed by consumer. Sender works in its own speed & consumer works in own speed, none of these EDA components black each other.
4) Agility: No point-to-point integrations. It’s easy to add new consumers to the system
5) Highly scalable and distributed: Tight coupling makes it much harder to add new functionality ,with loosely coupled components are by nature easily extended. Functionality/Feature ,New consumer can be added as just another independent consumer, having no impact on any other consumers in any way without breaking the existing functionality.
6) Cut costs: Rather than continuously checking everything happens on-demand . In this way, you’re not paying for continuous polling to check for an event. This means less network bandwidth consumption, less CPU utilization, less idle fleet capacity, and less SSL/TLS handshakes.
BTP Event Mesh reduces operation cost by providing real time visibility of communication between components
7) Register Once : No additional action needed by interested parties on significant change (event) in Producer
8) Flexible: Ability to scale individual components
9) High cloud affinity
10) Automatic updates
11) Connect Seamlessly: Provides reliable data transmission for extension and integration scenarios through decoupled communication.
12) High responsiveness
13) The service supports open standard messaging protocols [AMQP,MQTT,HTTP]
14) The service supports protocol agnostic libraries that can be used at the application configuration level for Java and Node.js
15) Fault Tolerance
- Setup SAP Event Mesh in BTP
- Integrating SAP BTP Event Mesh in CPQ
- SAP BTP Event Mesh Integration in SAP Cloud Integration (CI, CPI aka HCI) iflow using the AMQP Adapter : Blog https://blogs.sap.com/2019/11/20/cloud-integration-connecting-to-external-messaging-systems-using-the-amqp-adapter/
Generally, this step will be done by SAP BTP Subaccount Admin/Space Admin
- Event Mesh is available on the SAP BTP Trail Account as well with “dev” plan (instead of “default” plan)
- The Event Mesh trial service is only availble in AWS Europe (Frankfurt) and AWS US East (VA):
This setup will be configured by BTP Sub-Account Admin, as this setup is one time activity at BTP Sub-Account level. Align with BTP Sub-Account Admin if it is already setup
Follow this tutorial Get Started with the Event Mesh User Interface
- Follow this tutorialto create an instance of SAP Event Mesh with plan default.
- Alternatively follow one of the guides in SAP Help Portal.
Approach 1 (Centralized Approach)
By this time, you might know about A message client, which allows to connect to Event Mesh through its unique credentials to send and receive messages
Here Central Approach means à instead of creating Event Mesh Service instance & Message clients in Space you can have one central sub-account which provides Message Client Details ,Queue Details.. i.e in every space no need to create instance.
Cloud foundry allows for instance sharing to share a service instance between spaces and orgs. Instance sharing only supports sharing services between spaces and orgs in the same CF installation
The core of Event Mesh is always a part of a BTP CF subaccount, and it comes by default in an high-availability setup. A communication across subaccounts is in most cases technically feasible but can result in limitation right now (server binding not possible for example).
A message client allows to connect to Event Mesh through its unique credentials to send and receive messages
Approach 2 (Decentralized Approach / Manual setup)
The core of Event Mesh is part of a BTP CF subaccount, and it comes by default in an high-availability setup. Message Clients for SAP Event Mesh are created as BTP service instances in a space, you need to create an instance of SAP Event Mesh with plan default inside space.
You can manage Event Mesh Stuff (Queues, Topics ) from Event Mesh Dashboard easily
Note : In SAP BTP Trail Account ,Send a test event from the Event Mesh admin UI is not possible with the “dev” plan BTP Traill account
You can use Postman or any other HTTP client to send an event to the Event Mesh REST interface.
BTP Event Mesh in POSTMAN follow tutorial : https://developers.sap.com/tutorials/cp-enterprisemessaging-test-queue-sendreceive.html
Configuration in SAP Event Mesh for sending quote events from CPQ 2.0 Tenant to SAP BTP.
To enable SAP CPQ to send information to SAP BTP Event Mesh , you need to create subscriptions to events, which send notifications from SAP CPQ to SAP BTP Event Mesh
In order to create subscriptions to events, that is, to establish a connection between a triggered event and an BTP Event Mesh , follow the steps provided below.
Once you create a subscription, notifications will be sent to the specified queue in BTP Event Mesh whenever the events occur in CPQ
1) Go to Setup –> Providers –> Subscription to Events.
2) Click Add new Subscription to Event.
Name: Quote Created
Description: this field is optional.
Webhook URL (required): enter the URL of EMS Service Instance (ask your BTP Event Mesh Admin- Step1).
Webhook HTTP method (required): POST
Event name (required): QuoteCreated
Authentication type (required): oAuth
This should match the information from SAP BTP Event Mesh
Note: You can learn how to create subscriptions to events here
3) SAP BTP Event Mesh Integration in SAP Cloud Integration (CPI iflow) using the AMQP Adapter
1) Please refer Blogpost to design iflow https://blogs.sap.com/2019/11/20/cloud-integration-connecting-to-external-messaging-systems-using-the-amqp-adapter/
2) Here consumer application (receiver iflow) receives notification from BTP Event Mesh on Event triggered from CPQ Tenant.
Generally, EMS Designed to send maximum message size of 1 MB for all messaging protocols because EMS not designed to transport huge data between components.
if you observe consumer application payload (CI Iflow) received from Event Mesh has only few fields based on design of Producer API (in our case CPQ API)
So, Consumer applications in our case iflow has to initiate a synchronous call to fetch additional data from CPQ Tenant API
To Connect CI Tenant Iflow to CPQ Tenant using JWT Token approach you can below blog post then trigger second call to get additional data
To send this event forward to Microsoft Azure Event Grid and from there using the Event Handler Logic Apps to send out an Email
if you would like to Programmatically achieve this scenario “SAP BTP Event Mesh Integration with Microsoft Azure”
Implement EMS applications then follow blog post SAP BTP Integration with Microsoft Azure
#1) Implement BTP Event Mesh applications . You can refer Samples here
Refer: Events & Messaging https://cap.cloud.sap/docs/guides/messaging/
CAP provides intrinsic support for emitting and receiving events. This is complemented by Messaging Services connecting to message brokers to exchange event messages across remote services.
Using SAP Event Mesh in CAP https://cap.cloud.sap/docs/guides/messaging/event-mesh
CAP Application Receiving Events from SAP S/4HANA https://cap.cloud.sap/docs/guides/messaging/s4
Sending messages to SAP Event Mesh from a workflow: https://blogs.sap.com/2021/07/20/sending-messages-to-sap-event-mesh-from-a-workflow/
Triggering SAP BTP Workflow using SAP Event Mesh : https://blogs.sap.com/2022/01/24/triggering-sap-btp-workflow-using-sap-event-mesh/
You can Post questions ,get answers to technical challenges from SAP Event Mesh community