Technical Articles
Cloud Integration – Automated Notification for Critical or Exhausted JMS Resources
This blog describes how to send automated notifications (after the June-10-2018 update) in case the JMS resources are critical or exhausted using an integration flow.
Automated Notification for Critical or Exhausted JMS Resources
As mentioned in the blog ‘JMS Resources and Size Limits’ limited resources are available on the JMS broker. To get notified as soon as the resources get critical is required to take early actions to avoid runtime issues. This blog describes how to enable the alerting using a simple integration flow.
How to use the Alert Notification Service for Alerting with the OData APIs is described in the blog Automated Notifications via SAP Alert Notification Service for Critical or Exhausted JMS Resources.
Scenario Description
To enable the notification we use the OData APIs for JMS queue monitoring in an integration flow.. These APIs can be consumed via https://<APIUrl>/api/v1 where <tmn> is the address of the tenant management node(Neo) or the address of the API service instance (CF). Here we will use the API for the JMS broker monitoring via https://<APIUrl>/api/v1/JmsBrokers(‘Broker1’) .
Overall scenario looks like this:
The flow is triggered by a timer, fetches the JMS Resource data via OData adapter, evaluates the JMS resource status in a script and send a notification mail if the resources are critical or exhausted.
Let’s create the flow.
Create Integration Flow with Timer Start Event and OData Receiver
Create an integration flow with Start Timer Event. Using Request-Reply step call OData APIs to fetch details of the JMS broker via OData receiver channel. Below find the configuration of OData Receiver channel:
Note, that you need to use Operation Read(Get), enter JmsBrokers(‘Broker1’) as Resource Path and $expand=QueueStates as Query Options.
Read JMS Resources into Properties in Content Modifier
In a Content Modifier step you now read all the JMS resources via XPath into properties:
Create the following properties:
QueueCapacityOK: XPath to /JmsBrokers/JmsBroker/CapacityOk
QueueCapacityError: XPath to /JmsBrokers/JmsBroker/CapacityError
QueueCapacityWarning: XPath to /JmsBrokers/JmsBroker/CapacityWarning
MaxQueues: XPath to /JmsBrokers/JmsBroker/MaxQueueNumber
Queues: XPath to /JmsBrokers/JmsBroker/QueueNumber
Transactions: XPath to /JmsBrokers/JmsBroker/IsTransactedSessionsHigh
Providers: XPath to /JmsBrokers/JmsBroker/IsProducersHigh
Consumers: XPath to /JmsBrokers/JmsBroker/IsConsumersHigh
Capacity: XPath to /JmsBrokers/JmsBroker/Capacity
MaxCapacity: XPath to /JmsBrokers/JmsBroker/MaxCapacity
Evaluate Overall JMS Resource Status via Groovy Script
In a Groovy Script we now calculate the overall JMS resource status. Use the following script code:
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
def Message processData(Message message) {
// Get Properties
map = message.getProperties();
MaxCapacity = new BigDecimal(map.get("MaxCapacity"));
Capacity = new BigDecimal(map.get("Capacity"));
BigDecimal PercentCapacity = Capacity * 100 / MaxCapacity;
BigDecimal CriticalLimit = MaxCapacity / 100 * 80;
BigDecimal ExhaustedLimit = MaxCapacity / 100 * 95;
MaxQueueNumber = new BigDecimal(map.get("MaxQueues"));
QueueNumber = new BigDecimal(map.get("Queues"));
BigDecimal CriticalLimitQueues = MaxQueueNumber - 1;
if (Capacity >= ExhaustedLimit) {
CapacityStatus = "Exhausted";
}
else if (Capacity >= CriticalLimit) {
CapacityStatus = "Critical";
}
else {
CapacityStatus = "OK";
}
if (QueueNumber >= CriticalLimitQueues) {
QueueStatus = "Critical";
}
else {
QueueStatus = "OK";
}
int QueueCapacityWarning = new Integer(map.get("QueueCapacityWarning"));
int QueueCapacityError = new Integer(map.get("QueueCapacityError"));
if (QueueCapacityError >= 1) {
QueueCapacityStatus = "Exhausted";
}
else if (QueueCapacityWarning >= 1) {
QueueCapacityStatus = "Critical";
}
else {
QueueCapacityStatus = "OK";
}
int Consumers = new Integer(map.get("Consumers"));
if (Consumers == 1) {
ConsumerStatus = "Critical";
}
else {
ConsumerStatus = "OK";
}
int Providers = new Integer(map.get("Providers"));
if (Providers == 1) {
ProviderStatus = "Critical";
}
else {
ProviderStatus = "OK";
}
int Transactions = new Integer(map.get("Transactions"));
if (Transactions == 1) {
TransactionStatus = "Critical";
}
else {
TransactionStatus = "OK";
}
message.setProperty("PercentCapacity", PercentCapacity);
message.setProperty("CapacityStatus", CapacityStatus);
message.setProperty("QueueCapacityStatus", QueueCapacityStatus);
message.setProperty("QueueStatus", QueueStatus);
message.setProperty("ConsumerStatus", ConsumerStatus);
message.setProperty("ProviderStatus", ProviderStatus);
message.setProperty("TransactionStatus", TransactionStatus);
if ((CapacityStatus == "Exhausted") || (QueueCapacityStatus == "Exhausted")) {
message.setProperty("JMSStatus", "Exhausted");
}
else if ((CapacityStatus == "Critical") || (QueueCapacityStatus == "Critical") || (QueueStatus == "Critical") || (ConsumerStatus == "Critical") || (ProviderStatus == "Critical") || (TransactionStatus == "Critical")) {
message.setProperty("JMSStatus", "Critical");
}
else {
message.setProperty("JMSStatus", "OK");
}
return message;
}
Check Queue Capacity Status in Router
If you want to get additional information about dedicated message queues in case the queue capacity is critical or exhausted in a Router step check the Queue capacity Status and route to a script doing the detailed analysis for the dedicated queues that are critical or exhausted.
Define Non-XML Expression ${property.QueueCapacityStatus} = ‘OK’ for the branch going to the second Router. Default branch is the branch to the Filter step which is executed if the status is not ‘OK’. In the Filter you define an XPath to the //QueueStates to get the details for all the message queues available in the tenant. Note that you have to select Nodelist as Value Type.
In a Script step evaluate the message queues and return all critical and exhausted message queues as properties QueuesCriticalNames and QueuesExhaustedNames,
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import groovy.xml.*;
import java.util.*;
import java.text.SimpleDateFormat;
def Message processData(Message message) {
def body = message.getBody(String.class);
def SourceMessage = new XmlParser().parseText(body);
def QueuesExhausted = "";
def QueuesCritical = "";
SourceMessage.QueueState.each {
value = it.text();
state = it.State.text()
name = it.Name.text()
if ("1".equals(state)) {
QueuesCritical = QueuesCritical +' '+ name
}
}
SourceMessage.QueueState.each {
value = it.text();
state = it.State.text()
name = it.Name.text()
if ("2".equals(state)) {
QueuesExhausted = QueuesExhausted +' '+ name
}
}
message.setProperty("QueuesCriticalNames", QueuesCritical);
message.setProperty("QueuesExhaustedNames", QueuesExhausted);
return message;
}
Check JMS Status in Router
In the second Router step check the JMS Status and route to the mail receiver in case the JMS resources are critical or exhausted:
Define Non-XML Expression ${property.JMSStatus} = ‘OK’ for the branch going to the End event. Default branch is the branch to the Mail receiver which is executed if the status is not ‘OK’.
Configure Mail Receiver
Now we configure the Mail receiver channel sending out the notification. Configure the mail server and authentication you want to use. In the Mail Attributes define the mail receiver and sender mail address and the Mail Subject and Mail Body:
Subject: JMS Ressources ${property.JMSStatus}
Mail Body:
The JMS Resources on the Cloud Integration tenant are ${property.JMSStatus}.
Overall Queue Capacity Status: ${property.CapacityStatus} (${property.PercentCapacity} Percent)
Capacity Status for Queues: Error: ${property.QueueCapacityError} Queues; Warning: ${property.QueueCapacityWarning} Queues; OK: ${property.QueueCapacityOK} Queues
Queues with Warnings:${property.QueuesCriticalNames}
Queues with Errors: ${property.QueuesExhaustedNames}
Queue Status: ${property.QueueStatus} (${property.Queues} / ${property.MaxQueues} )
Transactions Status: ${property.TransactionStatus}
Consumer Connections Status: ${property.ConsumerStatus}
Provider Connections Status: ${property.ProviderStatus}
Deploy and Run the Integration Flow
Configure the Timer start event to run the flow every day or any other interval that suits your requirements. Deploy the integration flow.
Now you will get a Email as soon as the JMS resources get critical.
Hello Mandy
Great blog!
Is it possible also to get the name of the queue per API which is exhausted?
thanks and regards,
Markus
regards, Markus
Hello Markus,
currently this is not possible with the released APIs, we will check internally if we can enhance the API. the problem is that such queue-specific calls are much more expensive and are not supported in high frequency be the messaging system. Let's see what we can achieve.
Best regards
Mandy
Hi Mandy
Thanks allot for feedback and checking this topic:)
regards,
Markus
Hello Markus,
with the March update the API is extended so that you can also get the names of the queues in error/warning.
I will update the blog shortly before the change will get released.
Best regards
Mandy
Hi Mandy
Is the Update with the queue names already released to the customers?
Somehow we couldn't fetch the queue names at the moment.
regards,
Markus
Hello Mandy
Sorry forget my question. I mistaked:(
regards,
Markus
Hi Mandy
This are great news! Thanks alot!
regards,
Markus
HI Mandy,
Thanks for documentation. But i am getting error of 401 as you can see in the attachments.
I am using the Oauth2 client credentials.
Oauth2cred
i have also created the Security credentials and it is deployed.
So could you please help me to get the issue resolved?
Thanks in advance!
T&R,
Prabhat
Hello
the error means that something in the credentials is not correct. Please check carefully, maybe redeploy the credentials and restart the integration flow. If you do not get it resolved please open a ticket on LOD-HCI-PI-CON-OD.
Best regards
Mandy
Thanks Mandy Krimmel for the info but unfortunately, it is not working after changing and redeployment as well.
So i will raise a OSS ticket as suggested
T&R,
Prabhat.
Hi Prabhat. Was this solved? Facing same issue.
Hi.
Thanks, great info.
Is it predicted to add the message count and capacity together with the names of the queues in error/warning for a more refined monitoring?
Regards
Valter
Unfortunately this is not possible because those details cannot be retrieved easily as we are using eventing mechanism with is only able to provide basic details.
Those details would need to be checked by the alerted tenant admin via the queue monitor.
Best regards
Mandy
Hello Mandy
We are searching for the Data Store also a Monitoring / Alerting functionality , similar to the JMS Resources in this blog.
Is there something available on this topic?
We would be interested:
Unfortunetly we couldn't find any API which provides this kind of information.
thanks and regards,
Markus
Hello Markus,
there is not yet an API available to get similar information for the different Data stores. We have this on our ToDo list, but I cannot give you an estimation by when this would be provided.
Best regards
Mandy
Hi Mandy
Thanks for your feedback!
Good to hear that SAP already identifed this as a feature:)
Would be great if there will be a notification when the feature is available.
Thanks and regards,
Markus
Hi Mandy
i got the below error when i deploy the iflow :
[CAMEL][IFLOW][EXCEPTION] : org.apache.camel.FailedToCreateRouteException: Failed to create route Process_1 at: >>> SetProperty[QueueCapacityOK, xpath{java.lang.String}] <<< in route: Route(Process_1)[[From[direct:Automated_Notification_for_Cri... because of java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
[CAMEL][IFLOW][CAUSE] : Cause: org.apache.camel.RuntimeCamelException: java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
[CAMEL][IFLOW][CAUSE] : Cause: java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
this sounds like an error in your script, would suggest to check this.
Best regards
Mandy
thank you Mandy for your reply, I did double check the script many times but still it seems that there are something missing.
just to confirm do I need to import any sort of additional library to the script in your original post ?
below is my script :
Hello,
looks ok so far. Do you already get the error when you deploy the flow or in the monitor message processing? What is the status of the integration flow in the artifact monitor?
Best would be you create a ticket on LOD-HCI-PI-CON-SOAP and attach the integration flow.
Please also share the ticket here.
Best regards
Mandy
it failed to deploy the iflow (failed to start).
Question from where I can create this ticket ?
I added your script in my tenant and deployed without problem. There is no message processing log for this integration flow?
Another idea: check the declaration of the properties in the content modifier, maybe this is where the error is. Could you share a screen shot?
Thank you,
Mandy
here
hm, looks good. Could you also share the configuration of the OData channel to fetch the details?
Thank you
Mandy
message protocol: Odata V2
Hello ,
could you please change to Read(Get) operation in the OData channel instead of Query(Get)? See this comment in the blog:
--
Note, that you need to use Operation Read(Get), enter JmsBrokers(‘Broker1’) as Resource Path and $expand=QueueStates as Query Options.
Thank you
Mandy
still getting the same error :
[CAMEL][IFLOW][EXCEPTION] : org.apache.camel.FailedToCreateRouteException: Failed to create route Process_1 at: >>> SetProperty[QueueCapacityOK, xpath{java.lang.String}] <<< in route: Route(Process_1)[[From[direct:Automated_Notification_for_Cri... because of java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
[CAMEL][IFLOW][CAUSE] : Cause: org.apache.camel.RuntimeCamelException: java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
[CAMEL][IFLOW][CAUSE] : Cause: java.lang.ClassNotFoundException: /JmsBrokers/JmsBroker/CapacityOK
Hello,
please open a ticket on LOD-HCI-PI-CON-SOAP, attach the integration flow and error message
Thank you
Mandy
i did some digging and found the below service in SAP cockpit should i configure it ? if yes please provide me with required data to enter.
keep in mind that i did use the information available in the article but it gave me an error "400 bad request"
No, OData Provisioning is not required.
Are you running in CF? Then you need to activate the API plan to call the OData APIs, see: Setting Up Inbound HTTP Connections (for API Clients) | SAP Help Portal
Best regards
Mandy
i'm running on neo environment.
Could you please open a ticket on LOD-HCI-PI-CON-SOAP attach the integration flow and error message. It is really impossible to trouble shoot this via the blog comments.
Thank you
Mandy
Hi Mandy,
Thanks for this blog!
How about if we want to access fields like /QueueStates/Broker/IsTransactedSessionsHigh?
Querying using $expand=QueueStates returns us the overall status of JMS but whenever we try to access QueueStates sub-entity, we encounter Bad Request - An exception of type 'OData2Exception' occurred.
Using the same JmsBrokers(Key='Broker1').
Cheers,
Dan
Hello Dan,
please only use the described/released APIs. To get more details is not supported as this would raise too many calls to the broker. You need to stick to the released APIs.
Best regards
Mandy