How to tackle “disguised” errors in your integration flows

Before Your iFlows can be used in production, it is curcial to test them end-to-end

As I promised last time, I am going to elaborate on iFlow testing, today. You need to make sure that every new iFlow, that is to be put to work on your cloud platform integration tenant, can deal with the various inputs and create meaningful responses during message processing, as well as after the callback from the target system. I am going to focus on scenarios where there is a source and a target system because this is generally the case.

Possible error scenarios could be down to authorization or network problems, malformed web service messages and invalid values on the payload or downtime of one of the involved server components.

Let’s have a look in more detail at an invalid value scenario. Imagine you want to replicate a business partner from your source system to SAP Cloud for Customer (C4C) using a Cloud Platform Integration (CPI) iFlow. To do this you trigger the web service on your source system and for some reason the business partner does not show up on your customers view in C4C. What went wrong?

There are three places you can check to figure out where the problem lies.

What to check for

1. I’ll start at the beginning – the web service outbound monitoring of the source system. Did the message even leave the source system? This might fail due to expired certificates for example.
2. What about the message monitoring in SAP Cloud Platform Integration? Did the message fail? What does the stack trace show? To check that you need to raise the log level of the corresponding iFlow from none/info at least to debug or even to trace to check the output message. Since the latest CPI update you can access the trace messages also from WebUI (previously this was only possible from Eclipse). This feature is the most important troubleshooting tool within CPI.
3. If the CPI message monitoring is all green and the message looks promising, you need to continue with the web service inbound monitoring in your target system. What does the error payload in C4C show? Please note: recently I encountered errors in C4C, which were not even logged. This happens when the SOAP message leaving CPI was not formed correctly. So watch out for that one.

In this blog I would like to provide you with another set of tools to deal with possible errors. I call them “disguised” errors, because these messages show up as completed CPI messages, despite the target system highlighting that there’s an issue with the messages.

CPI treats messages with a HTTP return status code of 200 as successful. However, it is possible that the receiving system understood the request but couldn’t process it completely. In this case you will find errors and warnings on the payload from the target system’s response.

The screenshot above is from CPI message monitoring for our iFlow and it makes it look like a new customer was created on C4C using SOAP service manageCustomerIn, because the message status is “completed”. But the payload from the message trace shows the following:

Fig. 2 Payload of response from C4C for SOAP service manageCustomerIn

So “message completed” is a bit misleading, as the customer could not be created due to an invalid id number range. CPI considers this message completed, though, because the target system (C4C) received it and returned a status code of 200. But first things first.

Message validation makes your life easier

It would be better to avoid structural problems in your messages from the beginning. To do this, you can apply validation logic to ensure the message from the source system complies with your requirements. SAP offers a component to validate XML messages only. This means you need to convert your JSON or CSV payload to XML first, before you can apply any validations.

The standard validator component checks the XML content against a XML Schema Definition (XSD), which contains the XML structure definition, the permitted data types and the allowed value ranges. Often you can derive the XSD from the WSDL-file of the source and target system. This works for SOAP-based web services. For REST-based services such as OData and the likes you need to have a look at the documentation. It is also worth checking the $metadata-response of your OData service (https://<url>:<port>/yourOdatService?$metadata). However you won’t get any value ranges mentioned there. SAP offers public API descriptions here.

Possible validation exceptions should be handled by an Exception sub-process. Have a look here for example to get started with exception handling. Usually you end up with a failed message in CPI indicating a problem and ideally sending an email to the administrator. Exception details can be accessed with the variable ${exception.*}. Just add a content modifier to your exception sub process to create an error message, which can be emailed. To be more flexible we created a separate iFlow to send the email, because we are applying this kind of exception handling to all of our iFlows.

Fig. 3 Screenshot of message creation with details of exception

So far so good, but the message you saw earlier in fig. 1 does not produce an error!

Handling disguised error messages

Let’s take a look at how to handle this particular type of error message. The key to “detecting” the little devils is parsing the payload of the response from the target system (see fig. 2). In my example, today, the target system is Cloud for Customer.

package com.convista.groovy

import com.sap.gateway.ip.core.customdev.util.Message
import groovy.xml.MarkupBuilder

/**
 * C4C Error Message Parsing Exception
 */
class SAPMessageParsingException extends Exception
{
  public SAPMessageParsingException(String message)
  {
	super(message);
  }
}

def Message processData(Message message) {
	
	def body = message.getBody(java.lang.String);
	
	try {
		def isEmpty = new XmlSlurper().parseText(body).Log.Item.isEmpty();
		
		if(!isEmpty){
			message.setProperty("containsError", true);
		}else{
			message.setProperty("containsError", false);
		}
	} catch(Exception e){
		message.setProperty("containsError", true);
		throw new SAPMessageParsingException("XmlSlurper crashed while accessing the response payload with element [Log > Item] || MESSAGE from Super:"+ e.message);
	}

	//forward message
	message.setBody(body);
	return message;
}

Fig. 4 Groovy script example for parsing C4C response with error

The Script parses the message structure (see fig. 2) and searches for the tags <log> and <item>. In case the XmlSlurper finds log entries, I add a new property to the message to be able to work with that information. I intend to enhance the logic going forward. For now, recognizing the <Log> tag on the xml structure should be enough, because the completed messages look different.

Fig. 5 Screenshot of iFlow step using the property containsError finishing with message event

If the message property containsError is true, we trigger an Email containing the response from C4C to notify the administrator for further processing.

After that it is up to you to decide if you want to finish the iFlow with a message or an error event. An error event would highlight “disguised” errors better than the message monitoring on CPI does.

Ultimately, I envision a groovy script that identifies all kinds of “disguised” error messages for SOAP, REST and OData responses.

Furthermore, I would like to encourage you to create external parameters to be able to configure your logging activities within your iFlows. That way the logging can be switched on and off, on demand. As of today you can only do this in Eclipse, but SAP claims that support for the eclipse-based development environment will expire at the end of H1 2018. So eventually this feature should be available in WebUI too. Have a look here for some directions on how to add external parameters and here how to update your iFlow on WebUI.

Final Words

I have introduced you the various issues that play a role in monitoring the transmission of a web service message from the source to the target system through CPI, and what possible pit falls you need to watch out for. I have also introduced you to an error message type, that I call “disguised”, and I have  described a way you can handle them.

As always, please feel free to leave comments and ask lots of follow up questions.

Thanks

Martin