Cloud Platform Integration – Advanced logging in iFLows

Motivation:

The SCI runtime provides some logging / tracing capabilities for technical analysis and error handling out of the box. However, I always wished to have more insights into the results and outcome of the message processing.

So I developed a mechanism to write information into a separate logfile and attach that to the message during runtime. Thereby the outcome of the message processing can be easily obtained from the operations view perspective during runtime.

The general method can be used for various use cases and shall be seen as a pattern. The example scenario in this blog is an iFlow to retrieve material master data from SAP ERP and further process it (out of scope here). The iFlow should write the number of material data retrieved and log out the material numbers.

The starting point was the very good blog of Bhavesh Kantilal: HCI – Payload Logging using Groovy Scripts – Part 1

General approach:

1. I use a groovy script to create a String (log file) and store it as a message property during runtime.
2. At every point in the IFlow process you can add log messages to the log file by using groovy script.
3. After the IFlow process is completed, the log file is finally attached to the message log as an attachment.
4. In case of an exception (the exception might be logged as well) the IFlow is terminated exceptionally. So the log file must also be attached to the message log.

Runtime view:

In the CPI Message Processing Monitor the log file appears as attachment of the message:

The outcome of the iFlow is visible in the Message Monitor. The number of entities and the list of materialnumbers can be found in the log file.

Summary:

Pros:

• Logs are persisted permanently at the message and can be obtained as long as the message is stored in the HCI (30 days by default)
• You can log out any relevant information that you can retrieve in your groovy script (groovy is quite powerful here)

Cons:

• You add many additional steps (groovy scripts) to your IFLow – one for every log entry
• You should be careful to not log huge amount of data excessively because your HCI tenant has a limited amount of disc space available

Source code:

Here you can find the code of groovy script (the methods are called in the corresponding steps of the sample iFlow):

import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import javax.xml.xpath.XPathFactory;
import javax.xml.parsers.DocumentBuilder;
import javax.xml.parsers.DocumentBuilderFactory;
import java.io.ByteArrayInputStream;
import org.w3c.dom.NodeList;
import javax.xml.xpath.XPathConstants;

def Message initLog(Message message) {
    def map =  message.getProperties();
    // Logging
	java.lang.String logger = map.get("logFile");
	if (logger == null){
        logger = new String();
	}
 	logger += "Start the import process! " + System.lineSeparator();
	// Store the logfile as property on the message object
	message.setProperty("logFile",logger);
	return message;
}
def Message logInfo(Message message) {
    def map =  message.getProperties();
    def body = message.getBody(java.lang.String) as String;  
    // Logging 
    java.lang.String logger = map.get("logFile");
    if (logger == null){
       logger = new String();
    }
    // Gather the information to write into logfile
    // In this case: summary of payload from Odata	
    def xpath = XPathFactory.newInstance().newXPath()
    def builder     = DocumentBuilderFactory.newInstance().newDocumentBuilder()
    def inputStream = new ByteArrayInputStream(body.getBytes("UTF-8") );
    def doc     = builder.parse(inputStream);
    NodeList nodeList = (NodeList) xpath.compile("//materialnumber").evaluate(doc, XPathConstants.NODESET);
  
    String mat_list = new String();
    def length = nodeList.getLength();
    for (int i = 0; i < length; i++) {
                mat_list += nodeList.item(i).getTextContent() + ",";
            }
  
    logger += "Successully received materialnumbers: " + length + System.lineSeparator()  ;
	logger += "Received the following MATNR from ODAta:  " + mat_list + System.lineSeparator();
    // Store the logfile as property on the message object
    message.setProperty("logFile",logger);       
    return message;
}

/**
 * Stores the logFile as attachment of the message processing
 * It can be found in the message processing dashboard in SCI Message Monitoring
**/
def Message storeLog(Message message) {
    //Properties 
    def map =  message.getProperties();
    java.lang.String logger = map.get("logFile");
    if (logger == null){
       logger = new String();
    }
    logger += "END Import process " + System.lineSeparator();
    // Store the logfile as property on the message object
    message.setProperty("logFile",logger);       
    // Store LOG file on MSGLog 
	def messageLog = messageLogFactory.getMessageLog(message);
	if (messageLog != null) {
        messageLog.addAttachmentAsString("logFile " , logger, "text/plain");
	}
    return message;
}

def Message logException(Message message) {
       //Properties 
       def map =  message.getProperties();
       java.lang.String logger = map.get("logFile");
       if (logger == null){
           logger = new String();
       }
       logger += "EXCEPTION occured in processing Import " + System.lineSeparator()  ;
       // Store the logfile as property on the message object
       message.setProperty("logFile",logger);
       // Call methode to store Log
       storeLog(message);
}

I found the advanced logging very helpful to comprehend and monitor the message processing. I mainly use it to log out cumulative information of the process like the number of entities retrieved from an Odata service or the routing of the message during runtime. There are endless possibilities to use the logging and I hope it will benefit you. But please keep in mind to not log out sensitive business data, as it will be visible to individuals who can access the HCI runtime (data protection).

How do you manage logging of information? Any other suggestions?

Best regards,

Sven