Skip to Content
Technical Articles
Author's profile photo Showkath Ali Naseem

BTP Event Mesh Advance Concepts

SAP Event Mesh (aka enterprise-messaging) is a fully managed service offering which connects applications, services, and systems across hybrid landscapes.

This service allows components to communicate through asynchronous events and distributes business events from SAP and non-SAP sources across hybrid landscapes

The Event Mesh service is designed to publish/consume business events between backend systems and extension applications built on SAP BTP

So far you might know that in an event-driven architecture consists of event producers that generate a stream of events, and event consumers that listen for the events.

Publisher / Producer

 

A application/system/component  which will add message/request in the queue.

 

Consumer/Receiver

A application/system/component  which will receive/process the message from the queue

 

Queue

A queue is a storage  space used to host a message until it’s received by the subscriber.

Queue wont process message, it is first consumer & just store message/event.

 

You need to explicitly create a queue using Admin UI or Management REST API.

Topic

A topic is a space used to host a message for a specified period of time defined by the sender.

Queue is managed object but Topics are not managed. You can say that a topic is created on the fly when you start publishing messages to a topic, and destroyed when there are no consumers listening to that topic. If a publisher sends an event to a topic and there are no subscribers for this topic, then there is no storage for any period of time.

Queue Subscription A queue subscription allows a queue to subscribe to a topic for its message retention capability.
Message Client

A Message client is the central component in BTP EMS to define the access from / to Event Mesh, It also controls the access to topics and queues

 

 

1)  For Error handling what are options provided in EMS ?

 

From Producer Side you have below options

Request Header Values Description
x-qos

0

1

Quality of Service (QoS)

Quality of Service denotes the way the messaging protocol handles retransmission of the message and ensures its delivery regardless of network reliability.

 

The allowed values for specifying QoS are 0 and 1. The API for publishing messages requires a mandatory header x-qos. When the client calls the API for publishing messages with QoS 0, the service tries to deliver the message and returns an HTTP response with code 204 irrespective of whether the message is delivered. The best effort is made to deliver the message but, doesn’t guarantee that the message is sent to queue or topic.

When QoS 0 is used, the message is deleted from the queue without an acknowledgment from the client.

 

If a message client calls the API for publishing messages with QoS 1 for guaranteed delivery, then, the service responds with the HTTP response code 204. If the 204 response code isn’t received, it’s the client’s responsibility to retry until the response code 204 is received.  The message is deleted from the queue only after the acknowledgment is consumed

 

Use QoS=1, as you want the message to be processed and “acknowledged” by EM

As of now EMS providing a QoS=1, which means at least once. How to do that is dependent on the used protocol. Exactly once in order is not supported out of the box. This requires additional efforts on sender and on consumer side.

The QoS is a contract between sender and broker and is a second contract between broker and consumer, that’s one reason why this is not supported out of the box. More details on QOs https://help.sap.com/viewer/bf82e6b26456494cbdd197057c09979f/Cloud/en-US/3f424ff1ae3b4bc084c4f1ea0be96f54.html?q=qos

 

Time to Live

 

Attribute in Queue

With every publish request, the message client can set a time to live for the message in milliseconds. If the API is called without this header, the default value of 2592000000 (30 days) is used.

The Time-To-Live (TTL) value (in milliseconds) can be set for each guaranteed message published by a message producer. Endpoints can be configured to respect message TTLs through the REST APIs for management or the user interface. If message TTLs aren’t honored, the message is discarded by the endpoint

 

 

From BTP Event Mesh you have below options while configuring queue

 

Parameter Values Description

Max Redelivery Count

 

Default : 0
Minimum value : 0
Maximum value : 255

This parameter you can configure in Event Mesh Message Client à While creating/Updating Queue

 

1) For example, If you set Max Redelivery Count to 3, that means EM will “push” the message maximum 3 times to the consumer if the consumer actively rejects the message. Technically the consumer has 3 options: “Accept”, “Release” & “Reject”. This redelivery count is only increased on a “Reject”

If consumer “Release” in case of CI(aka HCI) -internal error, you’ll get that message inifinitly.

2) Either when the Max Redelivery Count is reached OR when the Max TTL is expired, the message will be removed from the queue. If you’d specify a Dead Message Queue (DMQ), it would be moved there.

Dead Message Queue

 

deadMsgQueue

deadMsgQueue  is the name of the Dead Message Queue (DMQ) used by the Queue. By default, each Queue is assigned a Default DMQ which does not exists. User must create a DMQ and assign it to the Actual Queue. when TTL expires for a message or the Number of redelivery attempts for a message exceeds the Max Redelivery value the messages instead of getting discarded can be moved to DMQ

EMS supports dead letter queues. The configuration needs to be done through the management API: https://help.sap.com/doc/75c9efd00fc14183abc4c613490c53f4/Cloud/en-US/rest-management-messaging.html#_queuep

* you can also use the management API .

https://help.sap.com/doc/75c9efd00fc14183abc4c613490c53f4/Cloud/en-US/rest-management-messaging.html#_queue

 

 

Max Time to Live

[in Seconds]

 

Default : 0

When Respect Time to Live              = ENABLED while creating/updating Queue in Admin UI Cockpit

Maximum time in seconds a message can stay in the Queue when Respect TTL option enabled. A message expires when the lesser of the sender assigned time-to-live (TTL) in the message and the maxTtl configured for the Queue, is exceeded. The expired mssage will move to Dead Message Queue (DMQ). A value of 0 disables expiry.

More information on help portal.

 

 

From Consumer Application

When QoS = 1 & If the 204 response code isn’t received, it’s the client’s responsibility to retry until the response code 204 is received.  The message is deleted from the queue only after the acknowledgment is consumed

 

If Receiver application is CPI Iflow then while configuring the AMQP Receiver Adapter you have below options in retry mechanism

 

Max. Number of Retries The default is set to 0, meaning there are no retries executed and the delivery status you defined for Delivery Status After Max. Retries is sent directly to the message broker. The maximum number of retries can’t exceed 99. If you do not specify the number of retries, endless retries are executed. In this case, the delivery status you defined for Delivery Status After Max. Retries will never be sent.

Define the number of retries to be executed before a different delivery status is sent to the message broker.

 

https://help.sap.com/viewer/368c481cd6954bdfa5d0435479fd4eaf/Cloud/en-US/99ce6748400b4dbfbadf633aeb111067.html

 

Blog Post

https://blogs.sap.com/2019/11/04/sap-cpi-retry-send-failed-asynchronous-messages-based-on-time-interval/

See this blog for a summary of the capabilities supported by the different brokers.

https://blogs.sap.com/2019/11/20/cloud-integration-connecting-to-external-messaging-systems-using-the-amqp-adapter/

 

Consume Expired Messages By default, this option is deactivated. Select if the adapter is to consume already expired messages.
Delivery Status After Max. Retries

Define one delivery status that is to be sent to the message broker after the maximum number of retries. Choose between the following options:

·         REJECTED

·         MODIFIED_FAILED_UNDELIVERABLE

 

 

 

2) Size & Limitations

 

Please check the limits on help portal and SAP Event Mesh (default plan) Size and Limits. Note that size restriction applies to request payload as well as headers.

For more descriptions of the messaging resources please check https://blogs.sap.com/2021/02/22/event-mesh-resource-units/

 

 

  • Question : Why The maximum message size is 1 MB for all messaging protocol ?

Generally, EMS Designed to send maximum message size of 1 MB for all messaging protocols because EMS not designed to transport huge data between components.

So, Consumer applications ( Example iflow )has to initiate a synchronous call to fetch additional data from CPQ Tenant API

 

3) How EMS Support   Performance improvements

In EMS Event-Driven Architecture (EDA)  Sender doesn’t need to ask the consumer all the time if there are processing resources available, or if some task is completed by consumer. Sender works in its own speed & consumer works in own speed, none of these  EDA components black each other.

For better performance in your EDA consumers should be quicker & available i.e  If consumer is always online then should not be message in queue .

Queue just store message so it is responsibility of Consumer in EDA to process message

 

4) Can two applications/consumers listen to Same Event Mesh Queue

Only one consumer receives any message from a queue.

By default, queues are created in NON_EXCLUSIVE mode, i.e. if there are more than one consumers to the queue, messages are delivered to each consumer in a round-robin fashion.

You may also create an EXCLUSIVE queue, in that case the first consumer on the queue receives all messages. If the first consumer disconnects, the second consumer starts receiving all messages, and so on.

More information on help portal.

5) Difference between Topic & Queue

# Queue Topic
1 Queue is a managed object, i.e. you need to explicitly create a queue using UI or Management REST API. Topics are not managed. You can say that a topic is created on the fly when you start publishing messages to a topic, and destroyed when there are no consumers listening to that topic.
2 Queues are persistent. When there are no consumers to a queue, messages are stored in the queue. Topics are not persistent. If your consumer to a topic is not running when a message is published to the topic, you’ll never receive that event.

 

6) Is Producer has to publish to Topic or Queue ?

 

If your ED architecture demands PUB-Sub model i.e one Produce & many consumers then Publish to Topic

Topic is for publish-subscribe scenarios, i.e. one-to-many messaging. Every subscriber to a topic receives all messages published to that topic.

Where as Queue is for point-to-point messaging. A message can be consumed by one and only one consumer.

 

7) What are best practises for Messaging Paradigms

Consider to always apply the following messaging paradigms to your messaging / eventing scenario.

  • Every Event in Publisher to be subscribe to One Topic,

For example

Event-A Publish to Topic-1,

Event-B Publish to Topic-2

i.e different Events to be Different Topic

  • Every Consumer in EDA listen to one Queue i.e  Create N queues for the N receivers.

For example

Queue-1 is for Consumer-1

Queue-2 is for Consumer-2

Create N queue subscriptions for the queues created above and assign them to the same topic name.

  • Publish messages to topics only: as soon as the messaging traffic leaves the tenant boundaries (e.g. Tenant/Reuse Traffic) each EM participant shall publish messages to well-defined topics only. P2P messages (send messages to queues directly) shall be avoided in any case. Reasons are the following:
      • Multicast of messages → it is up to the consumer to subscribe 1…n to a certain topic
      • Scaling flexibility → queues lead to a stickiness of consumers to the corresponding broker resource which would prevent EM to optimize the virtual broker
      • Clear Event/Message contract → topics are the basis to describe the event contract and events can be identified clearly through their topics in the future event catalog (Concept Eventing).
      • Non-existent consumers → the approach avoids to deliver messages to queues w/o attached consumers.
  • Publish to your own namespace: never publish to foreign namespaces
  • Consumption-driven subscriptions:the consumer defines how and how often a message is consumed

8) Example , Naming Conventions, Rules

In Publisher/Sender application, let’s assume you have below Events

  • OrderCreated
  • PaymentRecived
  • OrderConfirmed
  • Shipped
  • Refunded
  • OrderFailed

 

Dependents on your business & architecture ,the N number of consumers (Example : Application1,Application2 , CI-CPI-Iflow1,CI-CPI-Iflow1,) would like to receive notifications

So you can create N number of Queues i.e Every Consumer in EDA listen to one Queue. For example “Queue-1” for Consumer-1 similarly for “Queue-2 for consumer-2”

If one consumer application-1 intersted in 3 events , for example the can create 3 queues

 

<namespace>/OrderCreated

<namespace>/OrderConfirmed

<namespace>/OrderFailed

 

 

Queue Naming Conventions & rules :

 

The total length is limited to 100 characters.

All queue names must have a namespace as a prefix

The queue name must consist of one or more characters such as “A to Z”, “a to z” or “0–9 “, “.”, “-”, “_”, and “/”.

 

Topics : Example

You can create Topics as below

<namespace>/OrderCreated

<namespace>/OrderConfirmed

<namespace>/OrderFailed

 

or My recommendation would be:

<namespace>/Sales/Order/Created

<namespace>/Sales/Order/Confirmed

<namespace>/Sales/Order/Failed

 

Advantage is With this you could have queues subscribing to:

<namespace>/Sales/* —> would get all “Sales” related events

<namespace>/Sales/Order* —> would get all “Sales Order” related events

<namespace>/Sales/Order/Failed –> if your queue is only interested in “Failed Orders”

 

 

 

The syntax for topic names is as follows:

Topics must consist of one or more segments.

Forward slash “/” (ASCII 0x2F) must be used as a segment separator.

The segment separator must not appear at the beginning or ending of a valid topic.

The segments must consist of one or more characters such as “A to Z”, “a to z” or “0–9 “.

Up to 20 segments can be used.

Empty segments aren’t allowed.

The total topic length is limited to 150 characters.

 

 

The following are examples of valid topic names:

 

<namespace>/Production/Confirmation/Created

 

<namespace>/Process/Order/Released

 

<namespace>/Sales/Order/Created

 

 

The following are examples of invalid topic names:

//Production/Order/Released

<namespace>//Order/Released

<namespace>/Sales/Sales Order/Created

 

 

For Constraints Refer : https://help.sap.com/viewer/bf82e6b26456494cbdd197057c09979f/Cloud/en-US/72ac1fad2dd34c4886d672e66b22b54b.html

 

9 ) Is BTP Event Mesh Supports “Basic “ Authentication ?

 

Answer is ) No , EMS Supports only oAuth

10) What are options available in EMS to Delete Messages in Queue

A queue is a storage space used to host a message until it’s received by the subscriber.

Queue wont process message, it is first consumer & just store message/event.

So what are options available in cleaning

 

1) EMS Supports Timeout Handling for Message in the Queue

This option you can configure in Admin EMS  Web UI , while creating or editing queue

When Respect Time to Live              = ENABLED while creating/updating Queue in Admin UI Cockpit

Maximum time in seconds a message can stay in the Queue when Respect TTL option enabled. A message expires when the lesser of the sender assigned time-to-live (TTL) in the message and the maxTtl configured for the Queue, is exceeded. The expired mssage will move to Dead Message Queue (DMQ). A value of 0 disables expiry.

More information on help portal.

 

2) Purge Queue

In EMS Admin WebUI , Queue has “Purge Queue” option to delete all messages in Queue

11) Monitor Resources

 

EMS Admin WebUI has various option to configure Quota, Size of Resources.                                    UI also provides option to Monitor Consumption as shown below

12) What are Development ,Implementation Options in EMS

 

EMS Applications you can develop using protocol agnostic libraries that can be used at the application configuration level for Java and Node.js

 

Build Applications Using Java

 

To build messaging applications using Java, see Samples on GitHub for Java Information published on non-SAP site.

http://help.sap.com/disclaimer?site=https://github.com/SAP-samples/enterprise-messaging-client-java-samples

Build Applications Using Node.js

To build messaging applications using Node.js, see Samples on GitHub for Node.jsInformation published on non-SAP site.

http://help.sap.com/disclaimer?site=https://github.com/SAP/enterprise-messaging-client-nodejs-samples

Use REST APIs

To use the REST APIs provided with the service, see REST APIs for Development.

EMS Provides REST for Development , Testing .

Use REST APIs to Send and Receive Messages

Use REST APIs to Manage Queues and Queue Subscriptions

REST APIs for Messaging

https://help.sap.com/viewer/bf82e6b26456494cbdd197057c09979f/Cloud/en-US/dc18d8ebc226404c87db22e818f70145.html

 

 

Assigned Tags

      Be the first to leave a comment
      You must be Logged on to comment or reply to a post.