Unexpected ABAP pass-by-val method call

Prologue

I work a lot with pointers/references in ABAP, as I refuse to use CHANGING and EXPORTING method parameters. When using pointers you have to care about which data lies in which memory area. So, the first rule for working with pointers is: Don’t get pointers of stack variables. Or, if you do, make sure your pointers don’t find their way into the heap or to underlying stack layers.

If your variable lies on the stack and you grab a pointer, the pointer works as expected. But only until your method returns to its caller. Then the current stack layer is cleared/invalidated and your pointers are gone:

METHOD run.
	me->ref_test( ).

	WRITE me->class_ref->*. "Crashes

	"me->class_ref is not initial but points to freed
	"stack memory and is therefore not usable
ENDMETHOD.

METHOD ref_test.
	DATA(my_local_var) = 42.
	GET REFERENCE OF my_local_var INTO DATA(my_ref).

	me->class_ref = my_ref.

	WRITE me->class_ref->*. "Works
ENDMETHOD.

So, every time you grab a pointer and share it, make sure it points to the heap:

METHOD run.
	me->ref_test( ).
	WRITE me->class_ref->*. "Works, as the referenced variable
	                        "lies on the heap and wasn't cleared
ENDMETHOD.

METHOD ref_test.
	DATA my_ref TYPE REF TO i.
	CREATE DATA my_ref. "ABAP allocates heap memory for an int and
	                    "returns the pointer into my_ref

	me->class_ref = my_ref.

	WRITE me->class_ref->*. "Works
ENDMETHOD.

Pass-by-ref when calling methods

And, if you believe me or not: You’re very likely also using pointers in your code. Probably not directly, but as soon as you call a method, a function or a form with any parameter, you’re using pointers (except if you’re using the VALUE()-syntax in the method header).

Calling methods with parameters means copying values. Each time a method is called, there is stack allocated for all parameters and local variables of that method, and the parameters are copied into that stack layer.

Unlike other programming languages, where the values passed are really copied, so the called method can use them as local variables, ABAP passes parameters by ref by default. That means, it doesn’t copy the whole value, but grabs a pointer in the background and copies only that few bytes to the new stack layer. When using the passed value in the method, ABAP then transparently dereferences the pointer to make the parameter seem like a real variable. I believe, that that behaviour was introduced to compensate that internal tables are treated like they are lying on the stack. By passing the values by ref, big internal tables don’t have to be copied, but just their pointer is copied which is much faster while remaining fully usable in the called method.

Using that knowledge, we’re able to do things like:

METHOD run.
	DATA(my_line_ref) = me->get_line( me->huge_table ).

	WRITE my_line_ref->*. "Works
	"The ref points to some line inside of
	"me->huge_table and therefore to the heap
ENDMETHOD:

" --> table TYPE TABLE OF i
" <<< line_ref TYPE REF TO i
METHOD get_line.
	READ TABLE table INDEX 1337 REFERENCE INTO line_ref.
ENDMETHOD.

When calling get_line, we’re passing a huge table. But as parameters are passed by ref, the table isn’t copied and just a pointer is passed to the method. That means, table points to the same memory area me->huge_table does.

If we’re now reading from that table and save the reference of one line and return it, the caller can safely use that reference as it points to some line in table me->huge_table.

Main story

It was afternoon and I was about to use the remaining time to fix some bugs in a freshly developed program.
While programming and testing, my program crashed with error SYSTEM_DATA_ALREADY_FREE. That error occurs, when you try to dereference a pointer which points to freed/cleared/invalidated stack memory like in the first example. But I knew that I haven’t grabbed pointers to any stack variable as I’m paying attention to create all data, I want to share, directly inside the heap.

I followed the stacktrace and found a code with following structure as the origin of the error:

METHOD run.
	DATA(line_ref) = me->get_line( ).

	"Do something with line_ref
ENDMETHOD.

" <<< line_ref TYPE REF TO gtype_line
METHOD get_line.
	line_ref = me->read_line( me->get_some_object( )->some_table ).
ENDMETHOD.

" --> table TYPE TABLE OF gtype_line
" <<< line_ref TYPE REF TO gtype_line
METHOD read_line.
	READ TABLE table INDEX 1337 REFERENCE INTO line_ref.
ENDMETHOD.

As you see, get_line only combines read_line with an attribute of some object. The object is obtained by calling me->get_some_object( ) and from that object, the attribute some_table is accessed inline and passed as parameter to read_line. As parameters are passed by ref, I don’t have to worry about the ref returned by read_line as I passed a variable from heap and therefore the ref also points to the heap.

But the program crashes as soon as I’m trying to use line_ref in the run method. So I jumped into the debugger to look what’s going on there. It turned out, that the ref returned by get_line wasn’t bound and pointed to freed memory. Although not a single stack variable was involved!

So I debugged the application step by step to eventually find out, that the ref becomes unbound/the memory behind the ref becomes freed, when the read_line method returns.

But, that’s impossible! read_line uses a variable from heap to grab a reference! Does the garbage collector free heap memory even if there are references?
Well, no. The garbage collector does its job right and hasn’t removed anything. But why does my line ref become invalid when the method returns?

I did some changes to my code and found the one thing that changes the behaviour of ABAP to no longer invalidating my pointer:

" <<< line_ref TYPE REF TO gtype_line
METHOD get_line.
	DATA(some_object) = me->get_some_object( ).
	line_ref = me->read_line( some_object->some_table ).
ENDMETHOD.

When storing the object ref in a local variable first, and accessing the attribute using that var, everything works as expected and my ref remains bound and usable.

But, why? Using a variable containing some value or using the return value of a method makes no difference! Both times, the return value of the method is copied to the stack and then used. It’s exactly the same value. In this case, it’s both times the reference to the object returned by get_some_object.

While experimenting with this issue (and revealing another weird behaviour of ABAP), it always did the same thing: While being in the method which reads from a passed table, the reference is valid. When returning from that method, the ref immediately points to freed memory.

Everything pointed to one cause: The table I’m reading from, _must_ lie in the stack. And it _must_ lie in the stack layer of the method reading from it (read_line). But that would break the fundamental rule of ABAP method calling! Method parameters aren’t passed by val! They aren’t copied into the stack! And why does this behaviour only appear, when directly accessing attributes from returned objects inline, but not when accessing them through a variable?

In the meanwhile, a friend joined me to help searching the reason. I already left the office an hour ago, but I wanted to know the reasons and I already spent two hours on this little bug!

After testing another 30 minutes, we did the one test, that revealed, what’s happening:

METHOD get_line.
	DATA(some_object) = me->get_some_object( ).

	me->read_line( some_object->some_table ).
	me->read_line( me->get_some_object( )->some_table ).
ENDMETHOD.

" --> table TYPE TABLE OF gtype_line
METHOD read_line.
	DATA(obj) = me->get_some_object( ).
	INSERT VALUE( ... ) INTO TABLE obj->some_table.
ENDMETHOD.

While being in read_line, we modify the table that was passed to us. But we don’t modify it through the parameter table, but through the object obtained by me->get_some_object( ). As table was passed by ref, it points to the same memory area. That means, when modifying obj->some_table, we should see the change also in table.

And indeed, the first method call in which some_object->some_table was passed confirmed, that our test was working. The debugger showed that the table behind both variables table and obj->some_table growed as the table was passed by ref and it’s one single table which is referenced by both variables.

Then, we debugged the second method call which uses me->get_some_object( )->some_table and we waited for the debugger to again display the change of the table behind the both variables.

But, we got another result. While obj->some_table growed, table kept the state the table had before modifying it. And that means: table MUST be a copy of the original table! ABAP must have copied the memory area of the table! That also explained, why the reference becomes invalid as soon as the method is left. As the table table was copied, it’s now lying in the stack of the method, the pointer grabbed with READ TABLE also points to the stack. To a stack area, which gets cleared when the method returns to its caller.

And together with the previous test results, we were able to derive the following rule:

If an attribute is passed to a method in whose access-chain the return value of a method is used, the content of that attribute is passed by val instead of by ref

And yes, that breaks the fundamental rule of ABAP method calling. Well, now you know _when_ a value is copied, but it’s still a completely unexpected behaviour.

But, wouldn’t that mean, every time the return-value of a method is passed inline (instead of an attribute of a return-value) this behaviour is shown? Because, when using return values inline, the rule stated above should also apply, shouldn’t it?
Well, yes and no. Probably ABAP copies that value, but return values are always copied to the callers stack, which has to be done as the method returning the value lost its stack area so the return value has to be stored somewhere else. That’s the reason why you have to use RETURNING VALUE(...) in ABAP as it’s just not possible to internally return a pointer like for the IMPORTING-parameters. Other programming languages don’t have such a VALUE(...) syntax as they always copy both method parameters and return values. So, when using return-values directly as method parameters, it’s not noticed, whether the value is copied one time or two times. It only matters, if the method returns a pointer and that pointer is dereferenced directly. The pointer itself was the return value and therefore copied, but the data behind the pointer stayed the same. Even if the pointer was copied, it still pointed to the same memory area.

I haven’t any idea how to google that behaviour, therefore I’m asking you: Did you know that? Is that documented somewhere by SAP?

Test code

Below you’ll find the code I tested with so you can debug and reproduce the behaviour.

CLASS my_class DEFINITION.
  PUBLIC SECTION.
    DATA:
      table TYPE string_table,
      self  TYPE REF TO my_class.

    METHODS:
      get_me
        RETURNING VALUE(obj1) TYPE REF TO my_class,
      get_line
        IMPORTING tab         TYPE string_table
        RETURNING VALUE(ref1) TYPE REF TO string.
ENDCLASS.

CLASS my_class IMPLEMENTATION.
  " <<< obj1 TYPE REF TO my_class
  METHOD get_me.
    obj1 = me.
  ENDMETHOD.

  " --> tab TYPE string_table
  " <<< ref1 TYPE REF TO string
  METHOD get_line.
    "This line should modify both me->table and tab
    INSERT `new line` INTO TABLE me->table.

    READ TABLE tab INDEX 1 REFERENCE INTO ref1.
  ENDMETHOD.
ENDCLASS.


START-OF-SELECTION.

  DATA:
    ref1 TYPE REF TO string,
    obj1 TYPE REF TO my_class.

  CREATE OBJECT obj1.
  INSERT `line1` INTO TABLE obj1->table.

  ref1 = obj1->get_line( obj1->table ).
  "ref1 points to a string as expected

  "Yes, there is no sense in calling a method like 'get_me'.
  "it's just to have a method call in the access-chain to
  "trigger the behaviour
  ref1 = obj1->get_line( obj1->get_me( )->table ).
  "As soon as the return value of a method is used in the access-chain of
  "the parameter which is to be passed, its value will be copied to the stack.
  "ref1 is a freed reference and cannot be dereferenced anymore


  EXIT.