Handling multiple GET Requests in SAP CPI via OData Batch Request

In this blog post, I will explain an alternate approach to fetch data for multiple GET Requests within a SAP CPI IFlow. Therefore, in order to set the context, I have considered a case where input XML payload contains two user ids(shown below).

<?xml version="1.0" encoding="UTF-8"?>
<EmployeeList>
   <Employee>
      <ID>123456</ID>
   </Employee>
   <Employee>
      <ID>789101</ID>
   </Employee>
</EmployeeList>

The requirement is to fetch the full name for both the user ids. One of the commonly used approach is explained well in the blog.
https://blogs.sap.com/2019/04/01/creating-dynamic-queries-in-cpi-odata-receiver-adapter/

If there are only two user ids, the requirement above can be achieved by just configuring the SuccessFactors Adaptor in your iFlow and using $filter=userId eq ‘123456’ or userId eq ‘789101’
But in case of hundreds of user ids, data fetch via batch request option can also be utilized as explained below.

The batch request approach is illustrated below with an iFlow

Step-1: This step contains the input in the form of XML Payload containing user ids.

<?xml version="1.0" encoding="UTF-8"?>
<EmployeeList>
   <Employee>
      <ID>123456</ID>
   </Employee>
   <Employee>
      <ID>789101</ID>
   </Employee>
</EmployeeList>

Step-2: This step contains a Groovy script code which prepares a Batch Request containing multiple GET requests.

import com.sap.gateway.ip.core.customdev.util.Message;

def Message processData(Message message) {
  def body =  message.getBody(java.lang.String) as String
  def inputXml = new groovy.util.XmlSlurper().parseText(body)
  
  // Batch Payload
  String completePayload = "";
  
  // Generate Random Id to be used in Batch Code
  String batchId = UUID.randomUUID().toString();
  
  // Prepare Batch Code
  String batchCode = "batch_" + batchId;    
  
  inputXml.Employee.each { record ->
        // Get Employee Id
		def fieldValue="${record.text()}" as String
        
        // Prepare Payload for containing employee id in current loop pass
		String tempPayload = "--batch_" + batchId + "\n"
		tempPayload = tempPayload + "Content-Type: application/http " + "\n" + "Content-Transfer-Encoding:binary" + "\n" + "\n"
		tempPayload = tempPayload + "GET User('" + fieldValue + "') HTTP/1.1" + "\n" + "Host: host" + "\n" + "\n"
		
		// Add the above payload in complete payload
		completePayload=completePayload.concat(tempPayload)
	}  
	
	// Close the full payload
	completePayload = completePayload + "--batch_" + batchId + "--" + "\n"
	
	// Prepare Message Body
	message.setBody(completePayload)
	
	// Prepare Boundary Header Parameter to be added in message header
	String boundaryParam = "multipart/mixed; boundary=" + batchCode;
	
	// Prepare Message Header
	message.setHeader('Content-Type', boundaryParam)
	
    return message
}

The groovy code above will prepare a batch request as shown below:

--batch_09e378cd-55f5-4a83-81cf-f1280a6fd12e
Content-Type: application/http 
Content-Transfer-Encoding:binary

GET User('123456') HTTP/1.1
Host: host

--batch_09e378cd-55f5-4a83-81cf-f1280a6fd12e
Content-Type: application/http 
Content-Transfer-Encoding:binary

GET User('789101') HTTP/1.1
Host: host

--batch_09e378cd-55f5-4a83-81cf-f1280a6fd12e--

Step-3: This step contains an HTTP Adaptor which is being used for a Batch as shown below.

Step-4: This step contains the Groovy Script to parse the OData Batch Response from SuccessFactors Batch Call. This script will require two Archives to be added in Iflow resources

1. json-20150729.jar (or any higher version) – This Jar will be used to parse the JSON Object.
2. olingo-odata2-api-2.0.4.jar (or any higher version) – This Jar will be used to handle Batch Response

This step will prepare an XML Payload which will contain user ids and their corresponding Full Names (Check Step-5).

import com.sap.gateway.ip.core.customdev.util.Message;
import java.nio.charset.StandardCharsets
import java.util.HashMap;
import org.apache.olingo.odata2.api.client.batch.BatchSingleResponse
import org.apache.olingo.odata2.api.ep.EntityProvider
import org.apache.commons.io.IOUtils
import org.json.JSONArray
import org.json.JSONObject
import org.json.XML

def Message processData(Message message) {
    // Define StringWriter (Required to prepare XML OutputXml Payload)
    def writer = new StringWriter()
    
    // This is required while parsing the batch response received Remote API Call
    String contentTypeWithBatch = message.getHeader('Content-Type', String.class)
    
    // Read Response Body
    String responseBody = message.getBody(java.lang.String) as String
	
	// Read Read responseBody as InputStream
	InputStream inputStream = IOUtils.toInputStream(responseBody, StandardCharsets.UTF_8.toString());
	
	// parseBatchResponse as List
	List<BatchSingleResponse> parseBatchResponse = EntityProvider.parseBatchResponse(inputStream,
				contentTypeWithBatch);    
   
 	int i = 0
 	
 	// Loop all Responses
	while (i <= parseBatchResponse.getIndices().max()) {
	    
	    // Get Current Record From Response
		BatchSingleResponse batchSingleResponse = parseBatchResponse.get(i);
		
		// Get Employee Record in form of HashMap
		HashMap<String, String> parsedRecord = this.decipherSingleRecordBatchResponse(batchSingleResponse.getBody())
		if (!parsedRecord.isEmpty()) {
		   // Prepare XML Payload 
	 	   def xml = new groovy.xml.MarkupBuilder(writer).Employee { 
	 	       ID(parsedRecord.get("userId"))
	 	       FULLNAME(parsedRecord.get("fullName"))
	 	   }
		}
		i++
	}   
	
	// Prepare OutputXml
	String outputXml = "<EmployeeList>" + writer.toString() + "</EmployeeList>"
    
    // Set Body
    message.setBody(outputXml)
    return message
}    

static HashMap<String, String> decipherSingleRecordBatchResponse(String body) {
    // This HashMap will contain Employee Record containing userId and fullName
    HashMap<String, String> userIdsMap = new HashMap<String, String>()
    
    // Error HashMap
    HashMap<String, String> defaultErrorMap = new HashMap<String, String>()
    
    // Declare JSONObject
	JSONObject xmlJSONObj
	try {
	    // Read Body as a JSON Object
		xmlJSONObj = XML.toJSONObject(body);
		JSONObject jsonData= xmlJSONObj.getJSONObject("entry")
		JSONObject jsonDataContent = jsonData.getJSONObject("content")
		JSONObject jsonProperties = jsonDataContent.getJSONObject("m:properties")
		
		// Read First Name from JSONObject
		String firstName = getValue(jsonProperties, 'd:firstName')
		
		// Read Middle Name from JSONObject
		String middleName  = getValue(jsonProperties, 'd:mi')
		
		// Read Last Name from JSONObject
		String lastName  = getValue(jsonProperties, 'd:lastName')
		
		String fullName = new String()
		
		// User may or may not have a middle name
        if (middleName.contains('NOVALUE')) {
			fullName = firstName + " " + lastName
		} else {
			fullName = firstName + " " + middleName + " " + lastName
		}		
        
        // Prepare User Record in form of HashMap<String, String>
        userIdsMap.put("userId", jsonProperties.getBigInteger('d:userId').toString())
        userIdsMap.put("fullName", fullName)
        
        // Return Parsed Record of a user containing user id and employee full name
		return userIdsMap				
	} catch (Exception je) {
	    // return empty HashMap if there is an error with parsing
		return defaultErrorMap
	}		
	    
	return defaultErrorMap
}

static String getValue(JSONObject jObject, String propName) {
	try {
	    // Return Value of JSON Object
		return jObject.getString(propName)
	} catch (Exception e) {
        return 'NOVALUE'
	}
}

Step-5: This step contains a Content Modifier which is just capturing the response from Groovy Script in previous step as shown below.

<?xml version="1.0" encoding="UTF-8"?>
<EmployeeList>
   <Employee>
      <ID>123456</ID>
      <FULLNAME>ABHIJEET KUL</FULLNAME>
   </Employee>
   <Employee>
      <ID>789101</ID>
      <FULLNAME>AMIT KUMAR SHARMA</FULLNAME>
   </Employee>
</EmployeeList>

Advantages:

1. One of the main advantages of using this approach is that multiple GET requests on different APIs can be combined within same batch e.g. in this case, if it is required to fetch employee leave records along with their full names another GET request on EmployeeTime API can be added in the same Batch Payload.

2. Another advantage will be that using this approach will certainly reduce number of network calls made to the target system as multiple GET requests are combined in one Batch request and only one call is made to remote system to fetch data of multiple user ids.

Points to Remember:

1. There can be restrictions from the remote system on the number of requests that can be handled within one Batch Request (e.g. only 200 GET Requests in Batch are allowed). In this case, the iFlow shown above can be incorporated within a Splitter and Gather Step.
2. In case you want to know more about handling Odata Batch Request. Please check the link below.

https://www.odata.org/documentation/odata-version-2-0/batch-processing/