Skip to Content
Technical Articles

Conversion of XML to JSON using Generic Java Mapping

Update 10th July 2020 – Added functionality to convert numbers to String using the NUM_TO_STRING parameter.

 

Introduction:

I was recently trying to convert XML To JSON in the below format in SAP 7.4 and also trying to remove certain tags from the XML structure and force a few nodes as JSON Array.

[
{
  record : [
        { key1: value1 }
  ]
},
{
   record : [
        { key1: value1 },
        { key1: value2 }
  ]
} 
]

We can easily convert XML to JSON on the REST channel. However, there are many restrictions on it in SAP PI/PO.

Besides, in SAP PO 7.5 we can easily remove a key name by listing the tags in ‘setIgnoredElements’ on REST Receiver Channel.  However, this liberty isn’t available on SAP PI versions 7.4 & below. The Custom XML to JSON Conversion bean also doesn’t help much here, especially if you wish to remove a node from the JSON structure. Hence, I have implemented a Generic Java Mapping that can be used to achieve this and many other tasks.

This blog focus on XML to JSON conversion using Java Mapping and also provides the liberty to play around with the structure.

This can be used in any SAP PI/PO version.

 

Benefits of Using this Mapping :

  • Converts XML to JSON
  • Force convert JSONObject to JSONArray
  • Hide tag names in JSON structure
  • Delete key & value pair from a JSON structure
  • Crop up initial tags in the JSON structure
  • Convert the JSON root tag to Array
  • Convert Numbers to String

Examples are illustrated further below after the implementation section.

 

Implementation:

Step1:- Get the JAVA Mapping JAR file

You can directly download the zip file from here and unzip it.

This zip contains 2 JAR files. Import the 2 jar files in SAP ESR as Imported Archives.

Please note the Java mapping uses org.json.XML java library to convert XML to JSON, so if it’s not in your SAP library, you will receive missing ‘org.json.XML’ error. So do import the orgXmlJson jar file also.

 

Step 2:- Use the java mapping in your operation mapping

 

Step 3:- Define the 5 Import parameters in operation mapping.

  • ARRAY_NODES – Simple Type – xsd:string
  • DELETE_ENTRY – Simple Type – xsd:string
  • HIDE_KEYSSimple Type – xsd:string
  • LAST_LEVEL_TO_KEEPSimple Type – xsd:int
  • NUM_TO_STRINGSimple Type – xsd:string

Step 4:- Define Binding for the JAVA mapping in operation mapping as below:

 

Step 5:-  Assign values to these parameters in the Integrated Configuration as required

 

Step 6:- (Optional) If you are using REST channel, don’t forget to set the data as JSON in the Receiver channel

 

And you are done.

 

Understanding the parameters:

  • ARRAY_NODES:
    • This is of String data type.
    • All tags including the root tag that are to be converted into an array can be passed here as a list separated by commas.
    • E.g. ARRAY_NODES = Root,node1,node4
    • If you don’t want to convert any node to array in JSON, then keep the field blank.
  • HIDE_KEYS:
    • This is of String data type
    • If you wish to hide the tag name and pass only the equivalent values to JSON structure, then pass all the tag names as a list separated by commas to this field.
    • E.g. HIDE_KEYS = node2,root
    • If you don’t want to hide any keys in JSON, then keep the field blank.
  • DELETE_ENTRY:
    • This is of String data type
    • If you wish to delete a tag and its value from the JSON structure, then pass all such tag names as a list separated by commas to this field.
    • E.g. DELETE_ENTRY = node5,node6
    • If you don’t want to delete any entries, then keep the field blank.
  • LAST_LEVEL_TO_KEEP:
    • This is of integer data type
    • If you wish to delete the initial tags from XML hierarchy and retain the child nodes, then pass the level number to this field
    • Value 0 implies to not delete any level. (default value)
    • Value 1 implies removing the root node completely from the structure and keep its child node in the JSON structure.
    • E.g. LAST_LEVEL_TO_KEEP = 2 (This will remove first 2 levels from the XML/JSON structure)
    • If you don’t want to delete any entries, then set the field as 0.
  •  NUM_TO_STRING:
    • This is of the boolean data type.
    • If you want to covert all the numbers to string in the output JSON structure, then set this variable as ‘true’.
    • Setting this to true will result in all the numbers (integer, float, long, short, double) to be in double-quotes in the output JSON format.
    • Default value is ‘false’.

 

How to set the parameter value?

Now let’s understand how you have to set the parameter values. Below are a few examples that will help you understand the same.

Sample XML :

<ns0:Root_MT xmlns:ns0="https://namespace">
	<node1>
		<node2>
		<node3>abc</node3>
		<node4>xyz</node4>
		<node5>123</node5>
		</node2>
		<node6/>
	</node1>
	<node1>
		<node2>
		<node3>abc</node3>
		<node4>xyz</node4>
		<node5>200.005</node5>
		</node2>
	</node1>
</ns0:Root_MT>

 

 

Example1: If the default values of the parameters are set.

ARRAY_NODES = “”

HIDE_KEYS = “”

DELETE_ENTRY = “”

LAST_LEVEL_TO_KEEP = 0

NUM_TO_STRING = false

{ 	"Root_MT":
	{
		"node1": [
		{  "node2": {
				"node3": "abc",
				"node4": "xyz",
				"node5": 123
			},
			"node6": ""
		},
		{  "node2": {
				"node3": "abc",
				"node4": "xyz",
				"node5": 200.005
			}
		}
		]
	},
	"xmlns:ns0"="https://namespace"
}

 

Example2: Hide the root tag and send data as JSON Array. Also, delete the namespace tag from the JSON structure and set node5 as JSON Array

ARRAY_NODES = “Root_MT,node5”

HIDE_KEYS = “Root_MT”

DELETE_ENTRY = “xmlns:ns0”

LAST_LEVEL_TO_KEEP = 0

NUM_TO_STRING = false

{ 	[
	{
		"node1": [
		{  "node2": {
				"node3": "abc",
				"node4": "xyz",
				"node5": 123
			},
			"node6": []
		},
		{  "node2": {
				"node3": "abc",
				"node4": "xyz",
				"node5": 200.005
			}
		}
		]
	}
	]
}

 

Example3: Remove the level 1 i.e the root tag level, always keep node1 as JSON Array (even if one record is found)  and hide node1 key and delete the namespace attribute. Also, convert the numbers to string format.

ARRAY_NODES = “node1”

HIDE_KEYS = “node1”

DELETE_ENTRY = “xmlns:ns0”

LAST_LEVEL_TO_KEEP = 1   (As level count starts from level 0 being root)

NUM_TO_STRING = true

 [
	{  "node2": {
			"node3": "abc",
			"node4": "xyz",
			"node5": "123"
		    },
	    "node6": ""
	},
	{  "node2": {
			 "node3": "abc",
			 "node4": "xyz",
			 "node5": "200.005"
		     }
	}
]

 

Source Code:

If you are importing the code from the link above, you need not bother about the source code. But just for your curiosity purpose and understanding provided the source code below. Do have a glance at it and provide suggestions and feedback.

You can also find the code here.

package com.convert;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.UnsupportedEncodingException;
import java.util.Arrays;
import java.util.List;
import java.util.ArrayList;

import org.json.XML;
import org.json.JSONArray;
import org.json.JSONObject;

import com.sap.aii.mapping.api.AbstractTransformation;
import com.sap.aii.mapping.api.StreamTransformationException;
import com.sap.aii.mapping.api.TransformationInput;
import com.sap.aii.mapping.api.TransformationOutput;

public class XML2JSON extends AbstractTransformation {

	private List<String> array_nodes = new ArrayList<String>();
	private List<String> hide_keys = new ArrayList<String>();
	private List<String> delete_entry = new ArrayList<String>();
	private int last_level_to_keep = 0;
	private boolean num_to_string = false;

	public XML2JSON() {

	}

	public XML2JSON(List<String> array_nodes, List<String> hide_keys,
			List<String> delete_entry, int last_level_to_keep, 
			boolean num_to_string) {
		/*
		 * Declare constructor to define the class-level variables Input: 
		 * 1) array_nodes -> List to store Keys that should be JSONArray 
		 * 2) hide_keys -> List to store Keys that should be Hidden 
		 * 3) delete_entry -> List to store Keys, whose records are to be deleted from the structure 
		 * 4) last_level_to_keep -> integer to store maximum level of JSON Structure to be retained
		 * 5) num_to_string -> boolean variable to set if numbers are to be converted in string format
		 */

		this.array_nodes = array_nodes;
		this.hide_keys = hide_keys;
		this.delete_entry = delete_entry;
		this.last_level_to_keep = last_level_to_keep;
		this.num_to_string = num_to_string;
	}
	

	@Override
	public void transform(TransformationInput transformationInput,
			TransformationOutput transformationOutput)
			throws StreamTransformationException {

		/*
		 * This is the default method called by SAP PI Java mapping Input :
		 * InputStream and OutputStream objects
		 */

		// An info message is added to trace by calling the getTrace() method of
		// AbstractTransformation class
		getTrace().addInfo("JAVA Mapping to Convert XML to JSON Initiated");

		// Input Payload is obtained from transformationInput
		InputStream inputStream = transformationInput.getInputPayload()
				.getInputStream();

		// Input Parameters is obtained from transformationInput
		String array_nodes = transformationInput.getInputParameters()
				.getString("ARRAY_NODES");
		String hide_keys = transformationInput.getInputParameters().getString(
				"HIDE_KEYS");
		String delete_entry = transformationInput.getInputParameters()
				.getString("DELETE_ENTRY");
		int last_level_to_keep = transformationInput.getInputParameters()
				.getInt("LAST_LEVEL_TO_KEEP");
		boolean num_to_string = Boolean.parseBoolean(transformationInput.getInputParameters()
		.getString("NUM_TO_STRING"));

		// Display parameter value on trace
		getTrace().addInfo("Input Parameters listed below: \n");
		getTrace().addInfo("ARRAY_NODES : " + array_nodes);
		getTrace().addInfo("HIDE_KEYS: " + hide_keys);
		getTrace().addInfo("DELETE_ENTRY: " + delete_entry);
		getTrace().addInfo("LAST_LEVEL_TO_KEEP: " + last_level_to_keep);
		getTrace().addInfo("NUM_TO_STRING: " + num_to_string);
		
		// Output Payload Stream is obtained to send the data
		OutputStream outputStream = transformationOutput.getOutputPayload()
				.getOutputStream();
		
		// Convert Message Content type in Msg Header to application/json
		transformationOutput.getOutputHeader().setContentType("application/json");		

		// Instance of XML2JSON created and parameters passed to contructor
		XML2JSON obj = new XML2JSON(Arrays.asList(array_nodes.split(",")),
				Arrays.asList(hide_keys.split(",")), Arrays.asList(delete_entry
						.split(",")), last_level_to_keep, num_to_string);

		// Call readStreamContent() to Handle the input and output stream content
		try {
			obj.readStreamContent(inputStream, outputStream);
		} catch (UnsupportedEncodingException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
	}

	public void readStreamContent(InputStream inputStream,
			OutputStream outputStream) throws StreamTransformationException, UnsupportedEncodingException, IOException {

		/*
		 * This method Input: InputStream and OutputStream objects
		 */

		try {
			// Declare a String variable to store output json format
			String jsonPrettyPrintString = "";

			// If data is present in InputStream, it is stored in byte object
			byte[] buf = new byte[inputStream.available()];

			// Inputstream reads the data in byte format
			inputStream.read(buf);

			// A debug message is added to display the input XML
			if (getTrace() != null) {
				getTrace().addDebugMessage(
					"Input XML:\n" + new String(buf, "utf-8") + "\n ------");
			} else { // This section is added for normal JAVA Program run
				System.out.println(new String(buf, "utf-8"));
			}

			// Convert XML to JSON
			JSONObject xmlJsonObj = XML.toJSONObject(new String(buf, "utf-8"));
			// System.out.println("XML JSON ",xmlJsonObj.toString());

			// Call handleJSONData() to parse the JSON Structure to Delete a
			// record or convert it to an array
			if (array_nodes.size() > 0 || delete_entry.size() > 0)
				xmlJsonObj = handleJSONData(xmlJsonObj);

			// Convert the Output JSONObject to String
			jsonPrettyPrintString = xmlJsonObj.toString(2);

			// Remove the levels if required
			if (last_level_to_keep > 0) {
				jsonPrettyPrintString = (String) deleteLevel(xmlJsonObj, 0);
			} 

			// Hide Key names if required
			for (String text : hide_keys) {
				jsonPrettyPrintString = jsonPrettyPrintString.replaceAll("\""
						+ text + "\":", "");
			}  

			// Print the Final JSON structure in Trace
			if (getTrace() != null) {
				getTrace().addDebugMessage(
						"Output JSON:\n" + jsonPrettyPrintString + "\n ");
			} else {
				System.out.println(jsonPrettyPrintString);
			}

		
			// Convert the Output JSON Structure to bytes
			if (jsonPrettyPrintString == null
					|| jsonPrettyPrintString.length() < 1) {
				byte[] bytes = "{}".toString().getBytes("UTF-8");

				// Write output bytes to the output stream
				outputStream.write(bytes);
			}

			else {
				byte[] bytes = jsonPrettyPrintString.toString().getBytes("UTF-8");

				// Write output bytes to the output stream
				outputStream.write(bytes);
			}

		} catch (Exception e) {
			// Handle all exceptions
			if (getTrace() != null) {
				getTrace().addDebugMessage(
					"Exception while writing OutputPayload: IOException", e);
				outputStream.write("{}".toString().getBytes("UTF-8"));
				throw new StreamTransformationException(e.toString());
				
			} else
				e.printStackTrace();
		}
	}

	public JSONObject handleJSONData(JSONObject jsonObj) {
		/*
		 * Parse the JSON Structure to Delete a record or convert it to an array
		 * Input: JSONObject -> Json Sub structure to be updated Output:
		 * JSONObject -> Updated Json Sub structure with deleted records and
		 * arrays.
		 */

		try {
			// Create an array of keyset to loop further
			String arr[] = new String[jsonObj.keySet().size()];
			int k = 0;
			for (String key : jsonObj.keySet())
				arr[k++] = key;

			// Loop through all the keys in a JSONObject
			for (String key : arr) {

				// If there are records to be deleted, remove them and move to next key
				if (delete_entry.contains(key)) {
					jsonObj.remove(key);
					continue;
				}

				// If there are records to be converted to Array, convert it.
				if (array_nodes.contains(key)) {
					jsonObj = forceToJSONArray(jsonObj, key);
				}

				// If the sub node is a JSONArray or JSONObject, step inside the Object
				if (jsonObj.get(key) instanceof JSONArray) {
					JSONArray sjao = jsonObj.getJSONArray(key);
					for (int i = 0; i < sjao.length(); i++) {
						sjao.put(i, handleJSONData(sjao.getJSONObject(i)));
					}
					jsonObj.put(key, sjao);
				} else if (jsonObj.get(key) instanceof JSONObject) {
					jsonObj.put(key,handleJSONData(jsonObj.getJSONObject(key)));
				} else {
					// Convert number to String if num_to_string is set
					if (num_to_string){
						Object val = jsonObj.get(key);
						if(val instanceof Integer || val instanceof Float || val instanceof Double || val instanceof Long || val instanceof Short)
							jsonObj.put(key,jsonObj.get(key).toString());
					}
				} 
			}
		} catch (Exception e) {
			// Handle all exceptions
			if (getTrace() != null) {
				getTrace().addDebugMessage(
						"Exception while Updating Payload: ", e);
			} else
				e.printStackTrace();
		}

		return jsonObj;
	}

	public static JSONObject forceToJSONArray(JSONObject jsonObj, String key)
			throws org.json.JSONException {
		/*
		 * Force Convert a record to JSON Array Input: 1) JSONObject -> JSON Sub
		 * structure to be updated 2) key -> Key whose value is to be converted
		 * to JSONArray Output: JSONObject -> Updated Json Sub structure with
		 * deleted records and arrays.
		 */

		// Get the key value from JSONObject using opt() and not get(), as it
		// can also return null value.
		Object obj = jsonObj.opt(key);

		// If the obj doesn't exist inside my the JsonObject structure, create
		// it empty
		if (obj == null) {
			jsonObj.put(key, new JSONArray());
		}
		// if exist but is a JSONObject, force it to JSONArray
		else if (obj instanceof JSONObject) {
			JSONArray jsonArray = new JSONArray();
			jsonArray.put((JSONObject) obj);
			jsonObj.put(key, jsonArray);
		}
		// if exist but is a primitive entry, force it to a "primitive"
		// JSONArray
		else if (obj instanceof String || obj instanceof Integer) {
			JSONArray jsonArray = new JSONArray();
			jsonArray.put(obj);
			jsonObj.put(key, jsonArray);
		}
		return jsonObj;
	}

	public Object deleteLevel(JSONObject jsonObj, int current_level) {
		/*
		 * Based on the last level to be maintained, this method recursively
		 * deletes all previous levels. Input: 1) JSONObject -> Stores the json
		 * structure to be edited 2) Index -> Stores the level count to remove
		 * and is incremented as we step deeper in the levels Output: Object ->
		 * Currently returns String or JSONObject.
		 */

		// Read the first key in the JSONObject Structure
		String node = new ArrayList<String>(jsonObj.keySet()).get(0);
		// System.out.println("*" + node + "--"+
		// jo.get(node).getClass().getName());

		// Check if last level as per input is reached and return the output
		// string
		if (current_level == last_level_to_keep - 1) {
			// If instance is JSONObject, remove the key and return the output
			// string
			if (jsonObj.get(node) instanceof JSONObject) {
				return jsonObj.getJSONObject(node).toString(2);
			}
			// If instance is JSONArray, remove the key and return the output
			// string
			else if (jsonObj.get(node) instanceof JSONArray) {
				return jsonObj.getJSONArray(node).toString(2);
			}

			// If both above cases fail, invalid level was provided and return
			// empty string
			return "{}";
		}

		// If its not the last level step in deeper one level and pass Object
		// and incremented level index
		else if (jsonObj.get(node) instanceof JSONObject) {
			return deleteLevel(jsonObj.getJSONObject(node), ++current_level);
		} else if (jsonObj.get(node) instanceof JSONArray) {
			return deleteLevel(jsonObj.getJSONArray(node).getJSONObject(0),
					++current_level);
		}

		// If the object is no more an array or json object, invalid level was
		// provided and return empty string
		return "{}";
	}

}

 

Conclusion

This is how I handled my conversion using a generic and reusable custom java mapping that is able to handle XML to JSON conversion. Hopefully, this piece of code helps those who have faced similar situations.

Please do let me know if you have handled this situation in any other optimized way!!

Anyone who wishes to edit the content or convert this JAVA mapping to a custom XML to JSON conversion bean can please do the needful and share it…

And do hit the like button, if you found the blog useful..:)

 

Best Regards,

Kiran Roy

11 Comments
You must be Logged on to comment or reply to a post.
  • Hello Kiran,

     

    I tried the same approach and everything is working fine but in the output JSON for few fields values are not showing in string as it containing numbers. Could you please help how to handle for those fields.

     

    Regards,

    janardhan

    • Hi Janardhan,

       

      Updated the blog and program to add number to string conversion functionality. Hope it helps now. 🙂

       

      Regards,

      Kiran Roy

  • Hi

     

    I am passing ARRAY_NODES parameter value as ns1:MT_eINVOICE,Requestbody – whereas ns1:MT_eINVOICE is my root node name and Requestbody is my field name for array. I tried without ns1 as well but throwing same error. Tried with double quotes and without double quotes as well.

    All options failing with same reason.

    I am getting warning as “The input parameter ARRAY_NODES does not exist”.

    Can you please help how to fix it?

    • Hi Amarnath,

       

      You just have to pass RequestBody to ARRAY_NODES & that should work.

      ARRAY_NODES : RequestBody

      Check the example section of Blog for more clarity.

       

      Regards,

      Kiran Roy

  • Hello KIRAN Roy ,

    Wonderful blog. Very neat and upto point.

    I have a requirement where i have to pass final JSON(REST Receiver Channel) as an HASHBse64 converted string in REST header custom property.

    So was initially thinking to call REST adapter using UDF to get JSON file and then use UDF to HASH conversion to provide string to another REST adapter. But now i had 2nd option by which i can avoid duplication of rest channel. Will give it a go and provide feedback.

    Cheers

  • Hello KIRAN Roy ,

    Can you look into converting functionality into UDF?  Actually i am running into bit of problem as i have to put converted JSON under target XML structure.

     

    Regards

    • Hi Sukhdeep,

       

      It would not be possible to make it a part of UDF.

      But if I understand your requirement correctly, based on your previous comments, you want to encode data using BASE64 hashing before sending it to the receiver.

      So post JSON conversion, try applying the hashing adapter module as mentioned here.

      But to use this, you will have to install this custom adapter in SAP.

       

      Regards,

      Kiran Roy

  • Hi Kiran,

    I am getting the error as below

    Input parameter ARRAY_NODES does not exist in mapping program com/convert/XML2JSON .

    I have gone through all the examples but even for default value i.e.  ARRAY_NODES =”” the same error I am getting. Could you please suggest.

     

    Regards

    Kasturika Phukan

     

    • Hi Kasturika,

       

      Please perform the below checks… Either the file has not been downloaded correctly or you might have mistakenly missed some config step.

      1. To check if the file is downloaded correctly, please check if XML2JSON.class & .java files both are present and if you open the XML2JSON.java file, you should be able to see the full code… (Step 1)
      2. Ensure you have configured steps 3,4 & 5 correctly.

      In case everything is fine & the problem yet persists, kindly send a screenshot of the error message you are facing.

       

      Regards,

      Kiran Roy