Update on 28.01.2016

Recently SCN document PI REST Adapter – JSON to XML conversion (a part of series describing SAP REST adapter functionality and technical capabilities) has been updated and now contains information on enhanced XML/JSON conversion and a demo example on this topic.

SAP Help documentation (currently only found in help materials for release SAP NetWeaver 7.31) now contains description of enhanced XML/JSON conversion – refer to Converting XML/JSON – Advanced Adapter Engine – SAP Library.

Both references have been added to a section “Links” of this blog.

One of features of SAP standard REST adapter, is XML/JSON conversion – that surely makes sense, considering that internal processing in SAP PI/PO is done in XML format on one hand, and JSON format is de-facto format when dealing with REST architectural style on the other hand. Looking into recent SCN forum threads and questions raised about a REST adapter, it can be concluded that generation of JSON output for a processed XML message payload is not always clear and may be misleading. SAP actively enhances functionality of a REST adapter – customization and feature-enrichment of JSON processing being one of actively contributed areas. Many of such features have been documented in SAP Help materials, but one of quite powerful and flexible functionalities – namely, enhanced XML/JSON conversion – was only briefly mentioned in a SAP Note 2175218. In this blog, I would like to demonstrate usage of this functionality and provide details about valid parameterization.

Internally, a REST adapter makes use of 3rd party Jettison library for JSON processing tasks. In standard configuration, REST adapter relies on default conversion logic implemented in Jettison processor, which does not correlate or take into consideration payload elements properties as defined in a corresponding message type, but has its own optimization and type derivation mechanisms that are based on nature of the value of a processed XML document’s element rather than XSD schema of a processed message. As a result, this conversion may sometimes result in unobvious output – here are just few examples which are commonly faced:

  • If an XML element was defined as an array, but only contains one item in converted XML payload, Jettison processor will likely convert it to a non-array type;
  • If an XML element was defined as a String, but only contains numeric value in converted XML payload, Jettison processor will likely convert it to an integer type.

In some use cases, this kind of improper type conversion may be unacceptable – and this is where enhanced XML/JSON conversion parameterization helps solving a faced problem.

An idea behind enhanced XML/JSON conversion functionality introduced with the SAP Note 2175218, is explicit instruction of JSON processor on how to treat particular XML elements. Let us examine this functionality based on a practical example.

 

Below is definition of a message type used for a response message in a synchronous scenario, where we make use of a REST sender communication channel. As seen, it contains elements of various types, including an array:

Response message type.png

A sample response message in XML format looks like:

XML response.png

Using standard configuration of a REST sender channel, JSON formatted response message produced from an XML formatted message given above, is:

REST response - default.png

It can be noticed that some elements’ types were interpreted incorrectly – for example:

  • Element “ID” wasn’t recognized as a String, but as a number – Jettison processor treated it as a number, because element value contains only numeric characters;
  • Element “Properties” wasn’t recognized as an array – Jettison processor treated it as a non-array object with a nested structure, because an element contains only one child entry of an element “Property” (no other sibling elements “Property”).

Let’s fix this using enhanced XML/JSON conversion. In a REST sender channel, parameterization for enhanced XML/JSON conversion is done in a table “Custom XML/JSON Conversion Rules”. Below is configuration which aims troubleshooting type and conversion mismatches highlighted earlier:

REST channel - enhanced configuration.png

After executing an interface once again and checking JSON formatted response message, it can be observed that now JSON output is produced correctly:

REST response - with XML-JSON conversion enhancement.png

I didn’t find details regarding parameterization in official materials, so let me summarize acceptable and valid values used in enhanced XML/JSON conversion parameters, and explanatory notes regarding their usage, in a table below:

Field Description Valid values
XML Namespace XML namespace of an XML element
Prefix XML namespace prefix of an XML element
Name XML element name
Type

XML element type.

Following types are currently supported: String, Integer, Decimal, Boolean.

It makes no difference which notation for the type value is chosen as long as it is one of those mentioned in a list of valid values.

If no value is specified, no specific XML/JSON conversion instructions are applied and default logic of Jettison processor is applied.

String type

string

xs:string

xsd:string

Integer type

int

integer

xs:integer

xsd:integer

Decimal type

decimal

numeric

float

xs:decimal

xsd:decimal

Boolean type

bool

boolean

xs:boolean

xsd:boolean

Array Type

Indicator if an XML element is an array or not.

It makes no difference which notation for the array type indicator value is chosen as long as it is one of those mentioned in a list of valid values.

If no value is specified, array type indicator is set to “false” by default.

If element is array

1

true

yes

If element is not array

0

false

no

Default Value

Value that will be assigned to a JSON element in case XML/JSON conversion for a corresponding XML element fails.

For example, in a provided demo, value of an element “Quantity” will be processed as an integer. If original value cannot be converted to an integer (for example, it contains not only numeric characters, but its content is alpha-numeric), then JSON output will receive a default value for such an element, which is “0” in this case.

It should be noted that default value is not verified against element type specified in a field “Type” – it is treated as a String. In this way, for example, it is possible to specify default value “Invalid value” for an element “Quantity” in a provided demo. An error will not be issued neither during activation of a communication channel, nor at runtime during execution of a message by REST adapter, even though provided default value mismatches with an element type (integer). Having this in mind, attention should be paid to provided default value and its compliance to an element type in sake of consistency.

Any value.

Following values are treated specially:

“null”

(with quotation marks) – interpreted as String value “null”

null

(without quotation marks) – interpreted as null

“”

(just quotation marks) – interpreted as empty String value

Links

Please also refer to following materials discussing enhanced XML/JSON conversion in SAP REST adapter:

To report this post you need to login first.

26 Comments

You must be Logged on to comment or reply to a post.

  1. Eng Swee Yeoh

    Hi Vadim,

    Wow! Brilliant effort in unearthing such a hidden gem! Thanks for taking the effort to document down this feature (especially the parameter table) and sharing it!

    Regards,

    Eng Swee

    PS: I can’t view the first two images – not sure if something is missing there.

    (0) 
    1. Vadim Klimov Post author

      Hi Eng Swee,

      Thank you for your positive feedback! I’m glad if provided information will be helpful to others.

      You are right, for some reason, first two screenshots got corrupted. I re-inserted them, they should look better now.

      Regards,

      Vadim

      (0) 
      1. Eng Swee Yeoh

        Hi Vadim

        Thanks for updating the images. It looks fine now and gives a clearer picture of the example.

        This will definitely be useful to a lot of developers. It is blogs like this that really help bridge the gaps that are sorely missing in SAP’s documentation, and thus make life much easier for those of us working on PI 😉

        Kudos again for this!

        Regards

        Eng Swee

        (0) 
  2. Harish Babu

    Hi Vadim,

    Excellent blog! I am actually facing the same error as described in this Array and object issue in JSON, REST

    I am on PO 7.4 SP 12. I don’t have the table “Custom XML/JSON Conversion Rules” in my REST communication channel. So I need to do a patch upgrade according to the Note. I checked the note that you have referred in this, Note 2175218.

    It doesn’t have a patch level for SP12. The note provides patch only for SP 11 and SP 13.

    My necessity is to make a node an array even when it has only one record. So should I do a patch upgrade to 13 and then apply the patch mentioned in the Note 2175218.

    Regards

    Harish

    (0) 
    1. Vadim Klimov Post author

      Hi Harish,

      If you have a sandbox system, you may try applying most recent patches for installed SP12 and see if enhanced XML/JSON conversion functionality is contained in them and gets introduced to a system, or you may raise a request to SAP asking them if / which patches for SP12 contain this enhancement (maybe it turns out respective patch level was just missed in a list of relevant patches in a SAP Note).

      Regards,

      Vadim

      (0) 
      1. Ramesh P

        Hi Vadmin,

        Are the custom conversion rules available for PI REST Receiver channel as well?

        Currently our system are updated till SP11 and i dont see such options for receiver channel

        thanks

        Ramesh

        (0) 
        1. Vadim Klimov Post author

          Hi Ramesh,

          Yes, enhanced XML/JSON conversion functionality is available in both REST sender and REST receiver. If you already applied corresponding patch for SP11, which contains this feature (it shall be at least PL20 for SP11 for release 7.4), then please also ensure that adapter metadata has been updated, too (updated XI Content for software component SAP Basis has been imported in ESR). Refer to SAP Notes 1536986 and 1637955 for more details.

          Regards,

          Vadim

          (0) 
  3. Muniyappan Marasamy

    Thanks Vadim for sharing. I am able to use this feature after SP upgrade to 12.

    is there any was to log json data in the adapter audit logs? it will be handy for troubleshooting. we can take the same payload and test it from outside PI.

    Regards,

    Muni

    (0) 
    1. Vadim Klimov Post author

      Hi Muni,

      It is possible to log JSON message in case XML/JSON conversion is used. In REST communication channels, there is a configuration option for this (checkbox “Log JSON message” – it becomes available when XML/JSON conversion is enabled), which was introduced by new feature development described in SAP Note 2175263. JSON message logging functionality was introduced earlier by new feature development described in SAP Note 2079486. When this kind of logging activated, JSON message will be logged and visible in Message Monitor, in message versions. This will not log JSON message to an audit log, but will create a message version. I’m not aware of any SAP standard functionality of REST adapter that would enable logging JSON content into message audit log. But hopefully a logged JSON message in Message Monitor will fulfil your needs.

      Regards,

      Vadim

      (0) 
      1. Muniyappan Marasamy

        Thank you Vadim. i am able to view the json message. It is very useful for troubleshooting especially when exceptions thrown from rest service.

        Regards,

        Muni

        (0) 
    1. Vadim Klimov Post author

      Hello Harish,

      The feature described in this blog, fulfils requirement which was raised in the referred thread. The thread is relatively old and has not been active for already a long time.

      Regards,

      Vadim

      (0) 
  4. RAVIJEET DAS

    Hi Vadim,

    Is there a provision on the configuration where I can instruct the JSON to XML parser to skip certain JSON elements.

    In my JSON response I am getting a comments element which contains characters which are not permitted in XML, I wanted to skip this element completely.

    Below is the error I see in the REST receiver adapter:

    Information Server returned code: 200

    MP: exception caught with cause java.lang.RuntimeException:

    com.ctc.wstx.exc.WstxIOException: Invalid white space character (0x8) in text to output (in xml 1.1, could output as a character entity)

    Exception caught by adapter framework: com.ctc.wstx.exc.WstxIOException: Invalid white space character (0x8) in text to output (in xml 1.1, could output as a character entity)

    Thx in advance

    Ravijeet

    (0) 
    1. Vadim Klimov Post author

      Hello Ravijeet,

      The provided JSON does not look like well formed – no escaping for some characters contained in the value of the element ‘comments’, as well as it seems incomplete.

      As for your question, I’m not aware of any functionality / parameterization in REST adapter, where you can explicitly define ignored JSON elements. Instead, when the returned JSON is well formed document, can you test its processing once again and see if implicit escaping takes place.

      Regards,

      Vadim

      (0) 
      1. RAVIJEET DAS

        Hi Vadim,

        The JSON to XML parsing works good when I have no data in json element “comments”. When I was doing end to end testing in DEV with no data in comments the JSON to XML thing works good. Now in UAT I see data coming in comments field and this is all raw HTML data which they capture in this comments field. Now when I get the data in “comments” field, the JSON to XML fails in the REST receiver adapter.

        (0) 
        1. Vadim Klimov Post author

          Ravijeet,


          First of all, this kind of long extracts of the failed message for the particular scenario are not very appropriate for blog comments – this is a use case for a new forum thread.


          Regarding the error… As I wrote above, I don’t think you can explicitly ignore any specific JSON element when processing the response message in REST adapter by standard means. Can you check with the called system responsibles, if they can encode symbols so that they can be parsed further?


          Regards,

          Vadim

          (0) 
  5. yamini sri

    Hi Vadim,

    Nice blog. In my scenario REST adapter is configured as receiver channel. since my case a synchronous process from ECC(Proxy)–>Request–>PI–>REST SERVICE–>Response–>PI –>ECC. Please help out with the XML/JSON conversion in the receiver channel. Looking forward for your reply. 

    This is the request body which must be passed to my REST Service.

    (0) 
    1. Vadim Klimov Post author

      Hello Yamini,

      The message you provided as an example, corresponds to OData, but not pure generic RESTful. Hence, please check OData version used by the called service and based on that, verify if you can use OData adapter instead of REST adapter on PI side in your scenario.

      Regards,
      Vadim

      (0) 
      1. yamini sri

        Thanks Vadim ,for the clarifications. I am new to SAP PI. As a result of my findings i could add oData adapter by installing this add-on(SAP Process Integration, connectivity add-on 1.0) to the PI 7.4 System. Can you help me out with basic links to configure a oData receiver channel with the mapping structure. 

         

        (0) 
        1. Vadim Klimov Post author

          Yamini, please check official SAP documentation for configuration of OData receiver channels as well as materials on SAP Community such as blogs – there are already examples on how to use OData adapter in receiver flow scenarios. If after following them you will have any specific question, I would suggest you creating a separate question in Questions & Answers section – comments to the blog is not very appropriate place for having discussion on the topic that is not directly related to the subject of the blog.
          Regards,
          Vadim

          (0) 
  6. Avinash Ayanala

    Hi Vadim,

    Thanks for providing such valuable information for us.

    While tring to convert the message from json to XML format using rest adapter i am not able to get the attributes into XML structure from json(attributes are declare with ‘@’). Is there any option available for me to get it into XML.

    Thanks,
    Avinash
     

    (0) 
    1. Vadim Klimov Post author

      Hello Avinash,

      I don’t think current implementation of REST adapter supports XML attributes during JSON to XML conversion, have never seen any evidence of this.

      Workaround here could be either avoidance of XML attributes in the message type and their replacement with XML elements, or usage of intermediate message type. In the latter case, intermediate message type only has XML elements and not attributes and is used when converting incoming JSON document into XML document. Afterwards, you shall be able to make use of the generated XML document (containing only XML elements) and convert it to required final target message (containing XML elements and attributes) from within the mapping – precisely, making conversion of required XML elements into their counterparts / XML attributes.

      Regards,
      Vadim

      (0) 
  7. Abdul Gaffar ANSARY SHAIK

    Hi Vadim,

     

    In my scenario, we have a REST sender where I need to convert from JSON to XML.

    JSON:

    {
    “ReqType” : “C”,
    “AmendNo” : “0”,
    “RequestDate” : “20.01.2017”,
    “RequestTime” : “25”,
    DriverCode” : “3165”,
    “FirstName”: “XXX”,
    “LastName”: “XXX”,

    }

    Converted XML:

    <?xml version=”1.0″?>

    -<ns1:MT_Driver_C_Req xmlns:ns1=”urn:ZZZZ.com:DriverMaster”>

    <ReqType>C</ReqType>

    <AmendNo>0</AmendNo>

    <RequestDate>20.01.2017</RequestDate>

    <RequestTime>25</RequestTime>

    <DriverCode>3165</OOMCO_DriverCode>

    <FirstName>XXX</FirstName>

    <LastName>XXX</LastName>

    </ns1:MT_Driver_C_Req>

    The problem is I have an array of driver values where I wrapped the XML with a recordset which occurs 0-unbounded. How do i get this array type in request XML? I’m using Add wrapper Element in channel to add Message Type root node in the converted XML to make mapping work.

    (0) 

Leave a Reply