Customers often have questions on the best approach to reprocess failed messages in SAP Cloud Integration.
For example, you can use JMS Queues to asynchronously process messages as outlined in this blog.
Another approach is to manually reprocess the messages.
Integrations can fail for many reasons: bad data, the source or target system being unavailable or just a logic error in the integration. Customers often trigger an email as part of the exception handling routine in an iFlow to alert key users that the integration failed. Often, the source payload (and other relevant error info) is attached to the email so the key user can troubleshoot and correct the problem. Since the user is already in their email, reprocessing the messages can be done efficiently right from there.
The message can then be resent from the source system, the file placed back onto a server in a scenario where SFTP is used or manually corrected.
Sometimes, key users can't reprocess files due to restrictions on the servers (i.e. no SFTP access) or because the sending system can't resend the same message (i.e. retriggering an ASN from S/4HC).
A possible solution to allow users to reprocess messages is to have them send an email to an inbox with the original source payload (with corrected data, if required) that failed. Cloud Integration can then monitor an inbox using the delivered mail adapter and based on information in the message issue a process direct call to an iFlow for reprocessing.
The architecture used in this blog has one "main" iFlow that would have all of the processing logic. There would be an iFlow to receive the payload from the source system (via HTTPS call, SFTP poll, EDI VAN, etc.). An additional iFlow would be created to poll an email inbox for new messages to reprocess. Both of these iFlows would call the main iFlow via Process Direct call.
Before proceeding, here are two links that were the foundation for this blog and are excellent sources. The first is how to work with the mail adapter in Cloud Integration: Sender and Receiver Adapter in SAP Cloud Platform. The second link is a good blog on how to setup a Gmail account and connect to SAP Cloud Integration. Customers will often use their existing SMTP server for mail integration with CPI but I created a Gmail account for the purposes of this blog.
Let's take a look at the iFlow:
The mail adapter is configured as outlined in the blog linked above. You can configure the behavior such as deleting messages after processing or marking them as read and the scheduling interval on when to poll for new messages.
Several properties are set on the Content Modifier in this iFlow. For example, it may be a good idea to control who can send emails to this inbox since we don't want to process attachments from everyone. We wouldn't want any malicious user sending in sales order data that automatically would create the orders. Thus, we can compare the sender against this list (or make a call to an external source for determination). The message subject is also set to a property. This is useful if we wanted to have this iFlow call multiple scenarios. For example, if subject contains EDI850, call one iFlow and if it contains EDI856 call another iFlow.
The script steps retrieves the message subject, sets the process direct URL, checks for an attachment and sets it to message body, checks to make sure the sender is valid and sets the final property as to whether the request is valid or not.
The final step is a router check for p_isValidRequest equal to TRUE and then the Process Direct call to reprocess the message. If you are calling more than one iFlow, you can use the property set in the script to facilitate this call.
To see the iFlow in action,
You would want to add appropriate exception handling into this iFlow. And of course, this is just one of many variations how email can be used with Cloud Integration.
Marty
For example, you can use JMS Queues to asynchronously process messages as outlined in this blog.
Another approach is to manually reprocess the messages.
Integrations can fail for many reasons: bad data, the source or target system being unavailable or just a logic error in the integration. Customers often trigger an email as part of the exception handling routine in an iFlow to alert key users that the integration failed. Often, the source payload (and other relevant error info) is attached to the email so the key user can troubleshoot and correct the problem. Since the user is already in their email, reprocessing the messages can be done efficiently right from there.
The message can then be resent from the source system, the file placed back onto a server in a scenario where SFTP is used or manually corrected.
Sometimes, key users can't reprocess files due to restrictions on the servers (i.e. no SFTP access) or because the sending system can't resend the same message (i.e. retriggering an ASN from S/4HC).
A possible solution to allow users to reprocess messages is to have them send an email to an inbox with the original source payload (with corrected data, if required) that failed. Cloud Integration can then monitor an inbox using the delivered mail adapter and based on information in the message issue a process direct call to an iFlow for reprocessing.
The architecture used in this blog has one "main" iFlow that would have all of the processing logic. There would be an iFlow to receive the payload from the source system (via HTTPS call, SFTP poll, EDI VAN, etc.). An additional iFlow would be created to poll an email inbox for new messages to reprocess. Both of these iFlows would call the main iFlow via Process Direct call.
Sample Architecture to Reprocess Messages from Email
Before proceeding, here are two links that were the foundation for this blog and are excellent sources. The first is how to work with the mail adapter in Cloud Integration: Sender and Receiver Adapter in SAP Cloud Platform. The second link is a good blog on how to setup a Gmail account and connect to SAP Cloud Integration. Customers will often use their existing SMTP server for mail integration with CPI but I created a Gmail account for the purposes of this blog.
Let's take a look at the iFlow:
Email Polling iFlow
iFlow logic
- Sender channel polls Gmail inbox for new messages
- Initial Content Modifier sets properties
- Groovyscript checks mail for attachment, ensures that the sender is in the allowed senders list and that there is a valid URL to call via Process Direct.
- If all 3 conditions in step 3 are met, Process Direct call is made to the "main" iFlow. Otherwise, the email is ignored and deleted.
Sender Channel
The mail adapter is configured as outlined in the blog linked above. You can configure the behavior such as deleting messages after processing or marking them as read and the scheduling interval on when to poll for new messages.
Sender Channel Options
Content Modifier
Several properties are set on the Content Modifier in this iFlow. For example, it may be a good idea to control who can send emails to this inbox since we don't want to process attachments from everyone. We wouldn't want any malicious user sending in sales order data that automatically would create the orders. Thus, we can compare the sender against this list (or make a call to an external source for determination). The message subject is also set to a property. This is useful if we wanted to have this iFlow call multiple scenarios. For example, if subject contains EDI850, call one iFlow and if it contains EDI856 call another iFlow.
Content Modifier Properties
Script Step
The script steps retrieves the message subject, sets the process direct URL, checks for an attachment and sets it to message body, checks to make sure the sender is valid and sets the final property as to whether the request is valid or not.
def Message initialData(Message message) {
def body = message.getBody(java.lang.String) as String;
def messageLog = messageLogFactory.getMessageLog(message);
def pMap = message.getProperties();
def subject = pMap.get("p_subject");
if(subject == null){
subject = "none";
}
if(subject.contains("EDI850")){
message.setProperty("p_pdUrl", "/processOrder");
} else {
message.setProperty("p_pdUrl", "FALSE");
}
def startChar = "<";
def endChar = ">";
def sender = pMap.get("p_sender");
def email = sender.substring(sender.indexOf(startChar) + 1, sender.indexOf(endChar));
//read Attachment
Map<String, DataHandler> attachments = message.getAttachments()
if (attachments.isEmpty()) {
message.setProperty("p_validAttachment", "FALSE");
} else {
message.setProperty("p_validAttachment", "TRUE");
Iterator<DataHandler> it = attachments.values().iterator();
DataHandler attachment = it.next();
message.setBody(attachment.getContent());
}
if(pMap.get("p_approvedSenders").contains(email)) {
message.setProperty("p_isValidSender", "TRUE");
}
if(pMap.get("p_isValidSender").equals("TRUE") && pMap.get("p_validAttachment").equals("TRUE") && !pMap.get("p_pdUrl").equals("FALSE")) {
message.setProperty("p_isValidRequest", "TRUE");
}
messageLog.setStringProperty("ResponsePayload", "Printing Payload As Attachment")
messageLog.addAttachmentAsString("Email Payload sent:"+pMap.get("p_subject") + ", " + email + ", " +pMap.get("p_validAttachment"), body, "text/xml");
return message;
}Router and Process Direct Call
The final step is a router check for p_isValidRequest equal to TRUE and then the Process Direct call to reprocess the message. If you are calling more than one iFlow, you can use the property set in the script to facilitate this call.
Process Direct Call
To see the iFlow in action,
Email to Reprocess
Message Log
iFlow Execution
You would want to add appropriate exception handling into this iFlow. And of course, this is just one of many variations how email can be used with Cloud Integration.
Marty
- SAP Managed Tags:
5 Comments
