Concept idea for custom real-time data replication scenarios
SAP Business ByDesign (ByD) provides you with plenty of APIs and integration options and whenever possible SAP recommends to use standard OData APIs and Web Service APIs.
However, for some use cases a custom real-time replication of ByD data to some other application or system might be required, such that a newly created or changed business object instance (BO) in ByD shall trigger a real-time replication.
For my scenario, I assume that the start criteria to trigger the data replication are company specific and may even depend on extension fields. Additionally the data replication shall be triggered by data changes caused by users, web services and mass operations (for example mass data runs or mass changes).
A typical example could be a real-time replication of ByD master data to some external system.
The basic approach is to invoke an external web services embedded in the ByD business object logic using the SAP Cloud Applications Studio.
The challenge is to efficiently combine real-time replication with mass data operations and to decouple the external web service consumption from ByD user sessions.
Basically we could split the requirement in two scenarios for consuming external web services:
- Single instance event (for example a user changes single objects): The main requirement is a real-time replication, i.e. the external web service consumption has priority and available resources shall be used for this task.
- Mass instance event (for example user applies mass-change): The main requirement is resource optimization and to prevention bottlenecks due to peak workloads, i.e. the external web service consumption has no priority and system resource consumption shall not impact UI users.
Hence, the consumption of the external web service shall triggered immediately, but should be decoupled from the user transaction to avoid time delays from a user point of view while communicating to the external system.
Decoupling the web service call from the user session (i.e. introducing an asynchronous barrier) can be achieved in ByD using the SAP Cloud Applications Studio by two means:
- Asynchronous messaging
- Asynchronous background processing (Mass Data Run)
Asynchronous messaging is the better approach to support real-time replication requirements, but it may cause high system loads when huge number of messages will be created and bundling options are not used.
Hence the key question is: How to bundle business object changes for mass enabled data replication, while not loosing the real-time characteristic?
The idea is to use the SAP Cloud Applications Studio to create change notification and communication tasks in parallel.
The change notification acts as a filtered list of business object instances that shall be replicated. Using ABSL you have all flexibility to decide which changes of a business object trigger the creation of a change notification.
The communication task triggers the replication (i.e. invokes the external web service) and is decoupled from the user session by an asynchronous ByD internal communication. The communication task collects all open change notifications and decides on the number of business object instances that are send as bundle to the external system.
The ByD session ID would be a possible “Task ID” to group change notifications.
BODL/ABSL example to get the session ID:
- BO extension:
import AP.Common.GDT as GDT; … [Label("Session ID")] element SessionUUID : GDT:UUID;
- BO event Before save:
import ABSL; import AP.PlatinumEngineering as AP; this.Common.SessionUUID = SessionContext.GetSessionUUID();
What do you think about the concept idea? Any comments are welcome 🙂