*Note: This post is specifically targeted to my fellow SAP CPI developers who design SuccessFactors integration solutions and need a one stop shop for monitoring iflows (for developers/admins/managers) without having to go through SAP CPI logs and traces*
Being fairly new to the SAP CPI space, I usually am a bit more careful just to ensure our clients get not only the solutions they want, but the solutions they need (don't we all?). In which case it would be appropriate to mention how exasperating it has been on my end to provide them with an integration solution which shows them what they really need to see (the logs - appropriate success,failure and exception messages).
There have been a few amazing blog posts earlier on how to effectively log messages, like the one here by sfinn01 where he harnesses the power of groovy scripts to append messages to a log file during runtime and then adding the message as attachment once an iflow finishes the job. I found it to be the most reliable and easiest way an integration developer could trace what exactly happened for iflows that could execute for as long as 10 minutes (the trace functionality stops tracing data after first 10 minutes) or more.
On one hand where I found this approach to be the most suitable for me in most of the cases, I always wondered if we could provide our clients a one-stop-shop for monitoring their standard SuccessFactors integrations along with custom SAP CPI based integrations without even feeling the need to dive deep into the dark side of SAP CPI, the 'Monitor Message Processing' page.
While working on a project for one of our clients, I was suggested by my mentor to look into the Execution Manager framework and figure out if we could go that extra mile to give our customers what they exactly need to monitor the iflow execution, without having to go into SAP CPI to figure out where things went wrong.
Therefore, in this article I would first like to share my findings on Execution Manager and if it's possible to harness its superpowers in SAP SuccessFactors to further be able to use it in our SAP CPI iflows and make monitoring with SuccessFactors a lot simpler and tidier.
The fact that SuccessFactors welcomes custom integrations with wide-open arms, opens a whole new world of possibilities for monitoring the behavior of our custom solutions from within SuccessFactors instances itself. The Execution Manager in SuccessFactors is the one-stop-shop which provides a dashboard to give you an overview of all the custom and standard integration processes communicating with your SuccessFactors instance.
To navigate to Execution Manager Dashboard, simply type 'Execution Manager' in the search box on your SuccessFactors HomePage --> select 'Execution Manager Dashboard' from the options list and you'll be welcomed by a page similar to one shown below.
The beauty of Execution Manager is that SuccessFactors provides OData services for writing (logging) into the Execution Manager from an external middleware and takes care of how to display the data **bliss**.
Talking further about these processes, it is important to be able to log multiple messages during its execution. Therefore, if you click on the arrow displayed at the right hand side of each process, it further leads you to a more detailed view of the logs and activities (called 'Events') that have occurred during the execution, as shown below.
Each message logged as a part of a process is actually a 'POST' call to an entity called EMEvent (Entity Manager Event). Several such EMEvents carry a reference to a process (a single execution of a particular process) which resides in SuccessFactors as an instance of EMMonitoredProcess entity.
To create an EMMonitoredProcess, you need to create an EMEvent payload with the details of the EMMonitoredProcess you wish to create. Once a process is created, you can log several EMEvents for that process by including the details to the process that you created. For every EMEvent you create, SuccessFactors would group it within the process you've pointed to in the payload (further down the line in this series of articles, each such process would be created by an iflow execution in SAP CPI).
To make it easier, here's an Entity Relationship Diagram for your reference 🙂
Furthermore, for giving users more information about a particular message/event you can create EMEventAttribute(s) for every EMEvent.
Here's a sample payload to create an instance of an a EMMonitoredProcess, which has an Event (message) logged with an attribute.
*Note: make sure the date is a UNIX timestamp, in milliseconds.
Now let's try creating a process and add an event to it using postman first.
Great Success!
Now let's checkout what our Execution Manager Dashboard looks like:
A few things to take care of:
Now, after trying out the Execution Manager framework it is quite certain that logging from SAP CPI is just a matter of making a few calls from within your iflow to Execution Manager and you can log all you want for the developer (like yourself), administrators and managers, all at one place.
If this intrigued you and you wonder how can this be implemented within SAP CPI, checkout the 2nd episode of this series where I have shared my approach of using this framework along with Exception Sub-processes to ensure the messages from SAP CPI are logged in the Execution Manager without making the iFlows messy and scary to debug 😄
Please feel free to write below your thoughts, opinions and suggestions if you reckon there could be more to it or ways to make it more informative for people who could benefit from it.
I'd like to dedicate this article to my folks who helped me putting this article together:
My mentor chris.paine for finding the cure to my issues with monitoring, sarikagupta and wwong for pushing me deep into the Integration space while ensuring i'm not stuck anywhere in the middle and engswee.yeoh for motivating me to write about SAP CPI Monitoring and giving back to the community.
SAP, "SAP Knowledge Base Article," 02 03 2019. [Online]. Available: https://launchpad.support.sap.com/#/notes/2696270. [Accessed 10 03 2020].
Background
Being fairly new to the SAP CPI space, I usually am a bit more careful just to ensure our clients get not only the solutions they want, but the solutions they need (don't we all?). In which case it would be appropriate to mention how exasperating it has been on my end to provide them with an integration solution which shows them what they really need to see (the logs - appropriate success,failure and exception messages).
There have been a few amazing blog posts earlier on how to effectively log messages, like the one here by sfinn01 where he harnesses the power of groovy scripts to append messages to a log file during runtime and then adding the message as attachment once an iflow finishes the job. I found it to be the most reliable and easiest way an integration developer could trace what exactly happened for iflows that could execute for as long as 10 minutes (the trace functionality stops tracing data after first 10 minutes) or more.
On one hand where I found this approach to be the most suitable for me in most of the cases, I always wondered if we could provide our clients a one-stop-shop for monitoring their standard SuccessFactors integrations along with custom SAP CPI based integrations without even feeling the need to dive deep into the dark side of SAP CPI, the 'Monitor Message Processing' page.
While working on a project for one of our clients, I was suggested by my mentor to look into the Execution Manager framework and figure out if we could go that extra mile to give our customers what they exactly need to monitor the iflow execution, without having to go into SAP CPI to figure out where things went wrong.
Therefore, in this article I would first like to share my findings on Execution Manager and if it's possible to harness its superpowers in SAP SuccessFactors to further be able to use it in our SAP CPI iflows and make monitoring with SuccessFactors a lot simpler and tidier.
Execution Manager and Custom Integration
The fact that SuccessFactors welcomes custom integrations with wide-open arms, opens a whole new world of possibilities for monitoring the behavior of our custom solutions from within SuccessFactors instances itself. The Execution Manager in SuccessFactors is the one-stop-shop which provides a dashboard to give you an overview of all the custom and standard integration processes communicating with your SuccessFactors instance.
To navigate to Execution Manager Dashboard, simply type 'Execution Manager' in the search box on your SuccessFactors HomePage --> select 'Execution Manager Dashboard' from the options list and you'll be welcomed by a page similar to one shown below.
Fig 1: Execution Manager Dashboard
(courtesy: SAP SuccessFactors > Execution Manager)
- The navigation panel on the left hand side allows you to choose from standard (Integration Center), scheduled and custom integrations (Middleware Integrations).
- The main section of the dashboard shows all the 'Processes' being/already executed with their Process States, Time Stamps and other details related to the processes.
The beauty of Execution Manager is that SuccessFactors provides OData services for writing (logging) into the Execution Manager from an external middleware and takes care of how to display the data **bliss**.
Talking further about these processes, it is important to be able to log multiple messages during its execution. Therefore, if you click on the arrow displayed at the right hand side of each process, it further leads you to a more detailed view of the logs and activities (called 'Events') that have occurred during the execution, as shown below.
Fig 2: Events Logged in a Process
(courtesy: SAP SuccessFactors > Execution Manager)
Relation between Events (EMEvent) and Processes (EMMonitoredProcess)
Each message logged as a part of a process is actually a 'POST' call to an entity called EMEvent (Entity Manager Event). Several such EMEvents carry a reference to a process (a single execution of a particular process) which resides in SuccessFactors as an instance of EMMonitoredProcess entity.
To create an EMMonitoredProcess, you need to create an EMEvent payload with the details of the EMMonitoredProcess you wish to create. Once a process is created, you can log several EMEvents for that process by including the details to the process that you created. For every EMEvent you create, SuccessFactors would group it within the process you've pointed to in the payload (further down the line in this series of articles, each such process would be created by an iflow execution in SAP CPI).
To make it easier, here's an Entity Relationship Diagram for your reference 🙂
Fig 3: EMMonitoredProcess and related entities
(courtesy: SAP SuccessFactors > Integration Center)
Furthermore, for giving users more information about a particular message/event you can create EMEventAttribute(s) for every EMEvent.
Trying it out !
Here's a sample payload to create an instance of an a EMMonitoredProcess, which has an Event (message) logged with an attribute.
{
"eventName": "Begin iflow",
"eventType": "START",
"eventDescription": "My iflow just started",
"eventTime": "/Date(1581046477000+0000)/",
"eventAttributes": {
"__metadata":{
"uri": "EMEventAttribute"
},
"name":"details",
"value":"log a message and provide details"
},
"process":{
"processDefinitionId": "DiscoveryConsultingDemo",
"processInstanceId": "6ggfd9C6dbcf47648eefe633487af282",
"processType": "INTEGRATION",
"processDefinitionName": "DemoA",
"processInstanceName": "ProcessName"
}
}*Note: make sure the date is a UNIX timestamp, in milliseconds.
Now let's try creating a process and add an event to it using postman first.
Fig 4: Events Logged in a Process
(courtesy: Postman)
Great Success!
Now let's checkout what our Execution Manager Dashboard looks like:
Fig 5: Logged events for a process
(courtesy: SAP SuccessFactors > Execution Manager)
A few things to take care of:
- Make sure your ProcessInstanceId remains consistent for a group of EMEvents you wish to log under a single EMMonitoredProcess.
- The final value of the Process State is determined when event is either of type START, END or FAILED (see table below for different possible values of EMEvent and the corresponding Process State value).
EMEventType
final Process State
Use case A:
event 1- START
IN_PROGRESS
Use Case B:
event 1- START
event 2- INFO
event 3- END
COMPLETED_SUCCESSFULLY
Use Case C:
event 1- START
event 2- INFO
event 3- ERROR
event 4- END
COMPLETED_WITH_ERRORS
Use Case 😧
event 1- START
event 2- INFO
event 3- WARNING
event 4- END
COMPLETED_WITH_WARNINGS
Use Case E:
event 1- START
event 2- INFO
event 3- INFO
event 4- FAILED
FAILED
Table 1: EMEvent use-cases
Conclusion
Now, after trying out the Execution Manager framework it is quite certain that logging from SAP CPI is just a matter of making a few calls from within your iflow to Execution Manager and you can log all you want for the developer (like yourself), administrators and managers, all at one place.
Further Steps
If this intrigued you and you wonder how can this be implemented within SAP CPI, checkout the 2nd episode of this series where I have shared my approach of using this framework along with Exception Sub-processes to ensure the messages from SAP CPI are logged in the Execution Manager without making the iFlows messy and scary to debug 😄
Please feel free to write below your thoughts, opinions and suggestions if you reckon there could be more to it or ways to make it more informative for people who could benefit from it.
I'd like to dedicate this article to my folks who helped me putting this article together:
My mentor chris.paine for finding the cure to my issues with monitoring, sarikagupta and wwong for pushing me deep into the Integration space while ensuring i'm not stuck anywhere in the middle and engswee.yeoh for motivating me to write about SAP CPI Monitoring and giving back to the community.
References:
SAP, "SAP Knowledge Base Article," 02 03 2019. [Online]. Available: https://launchpad.support.sap.com/#/notes/2696270. [Accessed 10 03 2020].
- SAP Managed Tags:
12 Comments
