Skip to Content

Hi ABAP developers,

I would like to introduce a new open-source ABAP JSON library we have developed. Why the world needs a new JSON library? I will explain our rationale behind developing this library with its features. In the end it is about having more choices and knowing trade-offs. I would like to thank Medepia IT Consulting for letting this work become open-source with the MIT License.

Table of Contents:

  • Summary
  • Alternatives
  • Reasoning and features
  • Examples
  • Performance
  • Links
  • Warning
  • Conclusion

Summary

You can generate any custom JSON with this library unlike alternatives. Thus you can easily achieve API compatibility with another JSON server written in another language. Beside providing a serializer and a deserializer, this library defines an intermediate JSON class in ABAP. Further development may enable more JSON utilities based on this JSON representation.

Alternatives

CL_TREX_JSON_*

Standard transformation using JSON-XML:  https://scn.sap.com/community/abap/blog/2013/01/07/abap-and-json

Manual string manipulation: While it provides flexibility, it is tedious and error prone work. Sometimes it is used together with CL_TREX_JSON_*

These libraries also seek automatic mapping:

https://github.com/se38/zJSON/wiki/Usage-zJSON

https://wiki.scn.sap.com/wiki/display/Snippets/One+more+ABAP+to+JSON+Serializer+and+Deserializer

Reasoning and features

It is intriguing to me that there was no JSON node representation in ABAP. Let me give examples from other languages:

Working with JSON in dynamic or loosely typed languages is easier since easily modifiable representations for JSON object and array already exists in the standard language:

In strongly typed languages like ABAP, Java, Go there are two approaches:

Our library has chosen the intermediary representation approach defining the class ZCL_MDP_JSON_NODE.

/wp-content/uploads/2016/07/abap_json_node_988103.jpg

Features:

  • It provides flexibility down to JSON spec. This is important because you get the same flexibility as manual string manipulation without errors. So compatibility of your ABAP service or client with any other JSON API becomes possible without string manipulation.
  • You can deserialize any JSON string.
  • You can know exactly what deserializer will produce when you see a JSON string.
  • You don’t need to define intermediary data types just for JSON input/output.

Future ideas for development:

  • Intermediary ZCL_MDP_JSON_NODE class enables development of methods like JSON equality checker, beautification of JSON output, checks for spec validity for string and number values.
  • The library uses regexes for parsing. Most of the time regex can be a quick solution. However, I think finite-state machines are better suited for parsers in general.
  • We will work on this library based on our needs and your suggestions. For example, we can work towards 100% compliance with the JSON specification running edge case tests.

Examples

Examples here are in the shortest form to show how easy JSON manipulation can become. There will be more examples at the project repo using other features of the class. JSON node class is easy to understand if you study attributes and methods once.

Deserialization Example:

DATA:  l_json_string TYPE STRING.

CONCATENATE

‘{‘

‘ “books”: [‘

‘ {‘

‘ “title_original”: “Kürk Mantolu Madonna”,’

‘ “title_english”: “Madonna in a Fur Coat”,’

‘ “author”: “Sabahattin Ali”,’

‘ “quote_english”: “It is, perhaps, easier to dismiss a man whose face gives no indication of an inner life. And what a pity that is: a dash of curiosity is all it takes to stumble upon treasures we never expected.”,’

‘ “original_language”: “tr”‘

‘ },’

‘ {‘

‘ “title_original”: “Записки из подполья”,’

‘ “title_english”: “Notes from Underground”,’

‘ “author”: “Fyodor Dostoyevsky”,’

‘ “quote_english”: “I am alone, I thought, and they are everybody.”,’

‘ “original_language”: “ru”‘

‘ },’

‘ {‘

‘ “title_original”: “Die Leiden des jungen Werthers”,’

‘ “title_english”: “The Sorrows of Young Werther”,’

‘ “author”: “Johann Wolfgang von Goethe”,’

‘ “quote_english”: “The human race is a monotonous affair. Most people spend the greatest part of their time working in order to live, and what little freedom remains so fills them with fear that they seek out any and every means to be rid of it.”,’

‘ “original_language”: “de”‘

‘ },’

‘ {‘

‘ “title_original”: “The Call of the Wild”,’

‘ “title_english”: “The Call of the Wild”,’

‘ “author”: “Jack London”,’

‘ “quote_english”: “A man with a club is a law-maker, a man to be obeyed, but not necessarily conciliated.”,’

‘ “original_language”: “en”‘

‘ }’

‘ ]’

‘}’

INTO l_json_string

SEPARATED BY cl_abap_char_utilities=>cr_lf .

DATA: l_json_root_object TYPE REF TO zcl_mdp_json_node.

l_json_root_object = zcl_mdp_json_node=>deserialize( json = l_json_string ).

DATA: l_string TYPE STRING.

l_string = l_json_root_object->object_get_child_node( KEY = ‘books’

)->array_get_child_node( INDEX = 1

)->object_get_child_node( KEY = ‘quote_english’ )->VALUE.

START-OF-SELECTION.

WRITE: ‘Quote from the first book: ‘, l_string .

Serialization Example:

DATA: l_string_1 TYPE STRING.

DATA: l_root_object_node TYPE REF TO zcl_mdp_json_node

   ,l_books_array_node TYPE REF TO zcl_mdp_json_node

   ,l_book_object_node TYPE REF TO zcl_mdp_json_node

   ,l_book_attr_string_node TYPE REF TO zcl_mdp_json_node .

*Create root object

l_root_object_node = zcl_mdp_json_node=>create_object_node( ).

*Create books array

l_books_array_node =  zcl_mdp_json_node=>create_array_node( ).

*add books array to root object with key “books”

l_root_object_node->object_add_child_node( child_key = ‘books’  child_node = l_books_array_node ).

*You would probably want to do this in a loop.

*Create book object node

l_book_object_node = zcl_mdp_json_node=>create_object_node( ).

*Add book object to books array

l_books_array_node->array_add_child_node( l_book_object_node ).

l_book_attr_string_node = zcl_mdp_json_node=>create_string_node( ).

l_book_attr_string_node->VALUE = ‘Kürk Mantolu Madonna’.

*Add string to book object with key “title_original”

l_book_object_node->object_add_child_node( child_key = ‘title_original’ child_node = l_book_attr_string_node ).

l_string_1 = l_root_object_node->serialize( ).

*ALTERNATIVE:

DATA: l_string_2 TYPE STRING.

*DATA: l_root_object_node_2 type zcl_mdp_json_node.

*Create same JSON object with one dot(.) and without data definitions using chaining.

l_string_2 = zcl_mdp_json_node=>create_object_node(

)->object_add_child_node( child_key = ‘books’ child_node = zcl_mdp_json_node=>create_array_node(

  )->array_add_child_node( child_node = zcl_mdp_json_node=>create_object_node(

   )->object_add_child_node( child_key = ‘title_original’ child_node = zcl_mdp_json_node=>create_string_node(

   )->string_set_value( VALUE = ‘Kürk Mantolu Madonna’ )

   )

  )

)->serialize( ).

START-OF-SELECTION.

WRITE: / ‘string 1: ‘ , l_string_1.

WRITE: / ‘string 2: ‘ , l_string_2.

Challenge: Try doing these examples with CL_TREX_JSON_*

For more examples please visit GitHub repo.

Performance

On a test machine, using JSON string example above(l_json_string) deserializing and serializing again 10000 times takes 2.1 seconds on average. It shouldn’t have any performance problems with general usage. Complete benchmark code will be on the project repo.

DO 10000 TIMES.

  zcl_mdp_json_deserializer=>deserialize(

   EXPORTING json = l_json_string

   IMPORTING node = l_jsonnode ).

  zcl_mdp_json_serializer=>serialize(

   EXPORTING node = l_jsonnode

   IMPORTING json = l_json_string ).

ENDDO.

Links

Here is a presentation about this JSON library:

Medepia ABAP JSON Library ZCL_MDP_JSON

Project code repository:

GitHub – fatihpense/zcl_mdp_json: Medepia ABAP JSON library that can generate and parse any JSON string.

Warning

The library isn’t extensively battle tested as of now. Testing your use case before using it in production is strongly advised. Please report if you encounter any bugs.

Conclusion

If you are just exposing a table as JSON without much modification, it is easier and probably better to use CL_TREX_JSON_*

If you are developing an extensive application and if you want to design your API beautifully, this library is a pleasant option for you.

Thanks for reading.

Best wishes,

Fatih Pense

To report this post you need to login first.

19 Comments

You must be Logged on to comment or reply to a post.

    1. Fatih Pense Post author

      Thank you for the SAP note. So, CL_TREX_JSON* classes are intended for internal use only.

      As for the differences between this library and SAP transformation solution. Features of SAP transformation for JSON, and my personal thoughts are below:

      1. Standard and supported
        • ➕ This is certainly an advantage.
      2. Brings an extra layer of XML, which is necessary for utilizing underlying XML transformation structure.
        • ℹ If you like working with XML, this can be an advantage. Also you may use JSON-XML representation as a standard between two systems. However, performance effect of this extra layer should be researched. If they had created a JSON solution from scratch they probably wouldn’t have added the XML layer.
      3. ID Transformation (predefined default transformation, ABAP to JSON and back). It has the same logic as XML id transformation
        • ➕ If you are using this, then you want a standard JSON representation for your data. Then it is an advantage.
      4. XSLT or Simple Transformation for manual mapping. You should create a valid JSON-XML representation.
        • ➖ If you are using this, you want flexibility. Most of the time, using a class within your code to map fields should be way easier to develop and debug than using XSLT or ST. 

      In that solution they have chosen XML as an intermediary object. We have chosen an ABAP class. I think this class is better suited for complex and custom JSON scenarios.

      Best wishes,

      Fatih

      (0) 
      1. Sandra Rossi

        Thank you for this detailed feedback. For point 2, maybe Horst Keller can give us insights whether the JSON is based on the XML engine. Anyway, I doubt that an ABAP (interpreted byte code) solution is faster than a C (compiled) program, but you’re right that’s just a guess, this requires a performance comparison.

        (0) 
        1. Horst Keller

          As described in ABAP News for Release 7.40 – ABAP and JSON | SCN and in the documentation ABAP and JSON, the native support of JSON in ABAP is indeed based on the XML engine used by the sXML-library, where JSON-XML serves as an intermediate format. The idea was reuse of an existing performant and robust infrastructure instead of creating another one besides.

          That in fact means, that you cannot deal with JSON alone, but that for parsing and rendering you have to know the JSON-XML-fomat. For serializing and deserializing ABAP data you have to know the asJSON format (that can be mapped to asJson-XML).

          Regarding functionality I’d say, that JSON-writers and JSON-readers based on the sXML-library should allow you to perform all rendering and parsing tasks that you need.

          Regarding performance one should really test. One could take the examples given above, transfer them to standard parsing and rendering and compare the runtime.

          Since the sXML-Library is implemented by kernel modules, I tend to agree with Sandra Rossi, that the natve support should be faster than an ABAP only implementation. The question is, if ZCL_MDP_JSON_NODE is doing it from scratch or if it is a wrapper für ABAP’s native JSON support, that simply hides the JSON-XML-format.

          (0) 
            1. Fatih Pense Post author

              Horst Keller Thank you for clarification. I think utilizing an existing infrastructure is a good decision both from technical and from business perspectives.

              Sandra Rossi Thank you for benchmark. I was going to write it. Another result of the benchmark: You are faster than me 😀

              So in the light of new information, the advantages of this library can be:

              1. Manipulating JSON like XML DOM:
                • Cutting a part from one JSON and pasting under in another JSON is much easier in ZCL_MDP_JSON_NODE.
              2. In the scenario:
                • You want to render/encode a custom JSON or parse/decode from a custom JSON
                • And, you don’t want to use Simple Transformation or XSLT

                Sandra, you have used the library for benchmark. I have learned a few nice tricks from your code. If you have any suggestions for improvement regarding ease of use, methods etc. I will be glad to listen.

                In the future, I want to try finite-state machine parser instead of regex. Also maybe I will try adding optional JSON-sXML serializer/renderer backend (maybe I will even write my own kernel module for fun 😳 ).

                Thank you both for your interest and fruitful conversation.

                Best wishes,

                Fatih

                (0) 
                1. Marco Dahms

                  Hi Faith,

                  thanks for contributing to this blog community and sharing your JSON library.

                  A while ago I tried out all existing JSON libraries (all the ones that you mention above) available and encountered that most of them perform relatively slow when we deal with large volumes of data. I suspect this is due to the internal string handling (parsing and concatenating strings) in the ABAP layer. After some research I found that using the simple transformations (in particular I refer to identity transformations) the performance improved tremendously even with large data volumes. I believe this is the case because such transformations are executed inside the XML engine.

                  (0) 
                  1. Uwe Fetzer

                    This is true, because the transformation is part of the Kernel (C/C++ is always faster than ABAP 😉 ).

                    The addiditional libraries are only necessary if you have special needs (like lower case names, special date formats etc.)

                    (0) 
                    1. Marco Dahms

                      Hi Uwe,

                      thanks for the clarification.

                      I always wondered why using transformations we have to stick to upper case JSON keys. What is the cause of this restriction?

                      Thanks

                      Marco

                      (0) 
                      1. Sandra Rossi

                        I guess that’s only an arbitrary choice, but I think it’s best to say either everything lower case, or everything upper case, and not allowing exceptions (“except in deserialization any case is allowed”), especially when it’s about writing transformations for JSON, especially for Simple Transformations which are symmetrical.

                        (0) 
                      2. Horst Keller

                        See CALL TRANSFORMATION, where the facts are described:

                        Serialization

                        The case of the names in the XML or JSON data depends on how they are specified in the ABAP runtime environment. If specified statically (b1, b2, …), uppercase is used; if specified dynamically in stab, the case used there is used.


                        Deserialization


                        The case used in the XML or JSON data must match exactly the case specified in the ABAP runtime environment. If specified statically (b1, b2, …), uppercase is used; if specified dynamically in rtab, the case used there is used.


                        We had an discussion about that already in ABAP and JSON. Rüdiger Plantiko

                        pointed it out: How should ABAP know how to convert upper case to mixed? lower?  case.


                        Therefore, use your own transformations to adjust your data.



                        (0) 
                  2. Fatih Pense Post author

                    Hi Marco,

                    Thank you for your interest and support. I also think that slowness is caused by string operations. I searched for alternative options, but ABAP has a limited set of functions on this subject. If there was an string or byte stream interface between kernel and ABAP, any ABAP-only solution could be as nearly fast as C/C++ solution. (Interpreted languages can also be very fast!) Slowness increasing with size also suggests that underlying operations for string concatenation and substring is not suitable for this kind of algorithm.

                    Performance by size can be an interesting benchmark subject. I wonder at which size(kb,length) performance of ABAP-only libraries starts to become unacceptable.

                    On a side note, I learned that kernel modules are for internal usage. Even for research getting the source is difficult, otherwise I was eager to implement a module 😏

                    Finally I think in many scenarios JSON data is not that big, if you are using service as an API for user interface(citation needed 😳 ). And if you want flexibility this library provides another option with nice manipulation methods.

                    If you have any questions about this library, I will be glad to answer.

                    Best wishes,

                    Fatih

                    (0) 
    2. Attila Berencsi

      Hi Fatih,

       

      this is the second time I used your library in non-performance critical/non-mass processing objects. I can say, this is the most convenient library to use so far for my requirements. I do not need to fight with the pain, and I can realize injections really easily within 2 or 3 lines of coding. I can easiily access nodes where I already obtained a reference, and nested loops not needed. To be able to manage the JSON I need to use only node keys existing in the JSON documentat in real.  Many thanks for your great job again.

      Attila

      (1) 
      1. Fatih Pense Post author

        Hi Attila, I’m glad this library made your work easier! I think you are perfectly in the audience I had in my mind when I was writing the library. Maybe I should use your comment as “user testimonial” 🙂

        Thanks for your kind words, support & bug reports!

        Best wishes,

        Fatih

        (1) 
    3. Jack Zhou

      Hi Fatih,

      I think in the “manuel mapping” part, we can use the class cl_trex_json*, the model will be

      change, internal data object –>JSON—>JSON_NODE—>JSON.

      (0) 
      1. Fatih Pense Post author

        Hi Jack,

        I think cl_trex_json* directly converts to JSON string. If you accept the overhead of encoding/decoding two times, it can be used as a clever technique. In that case, zcl_mdp_json* will only give you the ability to edit your JSON structure in a flexible way.

        I have used the term “manual mapping” because with zcl_mdp_json* you have to decide which field in JSON string represents which field in your data object. There are alternative libraries that try to automatize mapping. However, I believe mandatory automatic mapping takes away expressiveness&power in the current solutions. There is a trade-off and you can use different libraries for different tasks. I haven’t found an elegant way to connect the parsing and automatic mapping yet. I’m open to suggestions.

        Also, cl_trex_json* is not supported by SAP as Sandra noted in the comment above. So it is for internal use and it shouldn’t be a base for future work.

        Thanks for your comment,

        Fatih

        (0) 

    Leave a Reply