Three different ways to serialize and deserialize complex ABAP data

If you’ve ever written an RFC-enabled function that transfers a structure, then you’ve probably seen a code inspector error like this:

“The type “BKPF” or the type of one of its subcomponents can be enhanced. An enhancement can cause offset shifts when the RFC parameters are transferred.”

So, what is that all about? Well, what it is saying is that the structure, BKPF in this case, can change. Your system can get upgraded, and one of the fields can get bigger or smaller. Or new fields can be added to the structure. And when that happens, if you forget to make the same change in the calling system, it may be unable to handle these “offset shifts”. And this could result in all sorts of unexpected behaviour. Like truncated fields, or values that end up in the wrong field. So, ideally we would like some way to make things more robust than that.

One way we can make things more robust is to ‘serialize’ data to a single string. So, for instance, we could replace the structure in our function interface with a single variable of type STRING. Let’s write a test ABAP program to demonstrate this idea:

report  z_xml_demo.
types: begin of ty_start,
         mychar(10) type c,
         mynum(5)   type n,
         myint      type i,
       end of ty_start.
types: begin of ty_fin,
         mychar     type string,
         mynum      type string,
         myint      type string,
       end of ty_fin.
data ls_start type ty_start.
data ls_fin   type ty_fin.
data lv_xml   type string.
ls_start-mychar = 'A1B2C3D4E5'.
ls_start-mynum  = 987654.
ls_start-myint  = 1234567890.
call transformation demo_asxml_copy source root = ls_start
                                    result xml lv_xml.
write / 'XML looks like this:'.
write / lv_xml.
call transformation demo_asxml_copy source xml lv_xml
                                    result root = ls_fin.
write / 'Final structure:'.
write / ls_fin-mychar.
write / ls_fin-mynum.
write / ls_fin-myint.

Note, the above code depends on the XML transformation DEMO_ASXML_COPY. If it isn’t on your system, you can create a transformation that is the same as DEMO_ASXML_COPY:

<?sap.transform simple?>
<tt:transform xmlns:tt="http://www.sap.com/transformation-templates">
  <tt:root name="ROOT"/>
  <tt:template>
    <node>
      <tt:copy ref="ROOT"/>
    </node>
  </tt:template>
</tt:transform>

Note that this is a “Simple SAP transform” type of transformation.

So, what does the XML produced by our example program look like? Well, it looks like this on my system:

<?xml version="1.0" encoding="iso-8859-1"?>#<node><MYCHAR>A1B2C3D4E5</MYCHAR><MYNUM>87654</MYNUM><MYINT>1234567890</MYINT></node>

In theory you could do something similar with your own hypothetical RFC function. If say you had an RFC function that had an exporting structure, and that structure had three fields or it had say a complex nested data structure, you could replace them with just a single string.

And how would things be more robust? Well if you look again at the example ABAP code, you’ll see that all of the fields in the final structure are of type STRING. This isn’t accidental, doing so means the code can accomodate changes in field length. Imagine say, that we made the fields in the final structure exactly the same as they were in the starting structure. So, say the “mynum” field also was a numeric field and had a length of 5. What would happen then if we ‘upgraded’ our design and decided that a length of 5 was too short and made the length in the starting structure 10, but we forgot to do the same in the final structure? Well, things wouldn’t work as ‘1234567890’ which would have come from our starting structure won’t fit when we try to squeeze it into a field in the final structure that only has a length of 5 digits! In fact, if you test this out, you’ll get a dump stating “Value loss during allocation”. Making all the final fields of type STRING means that they will always expand to accept the values given to them, and makes the solution more robust.

Another way in which using serialization makes things more robust is that it accomodates changes when new fields are added to the starting structure. So, even if we were to add a new field, “mynew” to the starting structure:

types: begin of ty_start,
         mychar(10) type c,
         mynum(5)   type n,
         myint      type i,
         mynew(15)  type c,
       end of ty_start.

Things would still work.

And if we were to remove a field from the final structure, for instance removing the “mynum” field:

types: begin of ty_fin,          mychar     type string,          mynum      type string,        end of ty_fin.

And if we messed with the sequence of fields? You guessed it, “things would still work!“

There is another benefit to serialization and that is that the calling system needn’t be an ABAP system – XML is a well known standard for data transfer between all kinds of systems.

Some folks (myself included) prefer JSON to XML, and there is even a page out there which calls it the “Fat-Free Alternative to XML”. Like XML, JSON has the advantage of being text-based and position independant. In ABAP the standard classes CL_TREX_JSON_SERIALIZER and CL_TREX_JSON_DESERIALIZER can be used for conversion between abap data types and JSON. You may not find CL_TREX_JSON_DESERIALIZER on older systems though – it can be found on a NW 7.30 system, but I can’t see it on my 7.0 system.

So, here is the earlier example written to use JSON that makes use of the above classes:

report  z_json_demo.
types: begin of ty_start,
         mychar(10) type c,
         mynum(5)   type n,
         myint      type i,
       end of ty_start.
types: begin of ty_fin,
         mychar     type string,
         mynum      type string,
         myint      type string,
       end of ty_fin.
data ls_start type ty_start.
data ls_fin   type ty_fin.
data lv_json  type string.
data lr_json_serializer   type ref to cl_trex_json_serializer.
data lr_json_deserializer type ref to cl_trex_json_deserializer.
ls_start-mychar = 'A1B2C3D4E5'.
ls_start-mynum  = 987654.
ls_start-myint  = 1234567890.
create object lr_json_serializer
  exporting
    data = ls_start.
lr_json_serializer->serialize( ).
lv_json = lr_json_serializer->get_data( ).
write / 'JSON looks like this:'.
write / lv_json.
create object lr_json_deserializer.
lr_json_deserializer->deserialize(
  exporting
    json   = lv_json
  importing
    abap   = ls_fin ).
write / 'Final structure:'.
write / ls_fin-mychar.
write / ls_fin-mynum.
write / ls_fin-myint.

The JSON output looks like this:

{mychar: "A1B2C3D4E5", mynum: "87654", myint: "1234567890 "}

The JSON is beautiful, don’t you agree?

Finally, what do you do if you need to serialize binary data? The other day, we had the requirement to send binary attachment data (in the form of XSTRING fields) between two systems via an RFC call. And we could have had more than one attachment, so we were working with an internal table of XSTRING fields. In this case, because the data was binary an alternative approach to using XML or JSON had to be used.

The ABAP command:

export lt_complex_table_containing_xstrings to data buffer lv_xstring.

was used for our serialization, and similarly deserialization was achieved by the command “import from data buffer”. Note that the serialized field in this case is of type XSTRING and not STRING. Also, the documentation for this command does state that “the undefined content of alignment gaps in structures can result in different data clusters with structures that otherwise have the same content”, so this method of serialization is not as fault tolerant of field position. However, it worked flawlessly for our data in an internal table that could have had one or fifty rows.

So there you have it: three different ways to serialize and deserialize complex ABAP data: XML, JSON and EXPORT TO DATA BUFFER.

My challenge to you is to describe all the other methods that you know about in the comments below so that we’ve got a comprehensive list in one place!

Update:

I have been informed that I’ve been serializing to JSON the old way. JSON is now supported natively in ABAP and support has been added to the sXML library for it. See: http://scn.sap.com/people/horst.keller/blog/2013/01/07/abap-and-json