Unexpected ABAP pass-by-val method call
I work a lot with pointers/references in ABAP, as I refuse to use CHANGING and EXPORTING method parameters. When using pointers you have to care about which data lies in which memory area. So, the first rule for working with pointers is: Don’t get pointers of stack variables. Or, if you do, make sure your pointers don’t find their way into the heap or to underlying stack layers.
If your variable lies on the stack and you grab a pointer, the pointer works as expected. But only until your method returns to its caller. Then the current stack layer is cleared/invalidated and your pointers are gone:
METHOD run. me->ref_test( ). WRITE me->class_ref->*. "Crashes "me->class_ref is not initial but points to freed "stack memory and is therefore not usable ENDMETHOD. METHOD ref_test. DATA(my_local_var) = 42. GET REFERENCE OF my_local_var INTO DATA(my_ref). me->class_ref = my_ref. WRITE me->class_ref->*. "Works ENDMETHOD.
So, every time you grab a pointer and share it, make sure it points to the heap:
METHOD run. me->ref_test( ). WRITE me->class_ref->*. "Works, as the referenced variable "lies on the heap and wasn't cleared ENDMETHOD. METHOD ref_test. DATA my_ref TYPE REF TO i. CREATE DATA my_ref. "ABAP allocates heap memory for an int and "returns the pointer into my_ref me->class_ref = my_ref. WRITE me->class_ref->*. "Works ENDMETHOD.
Pass-by-ref when calling methods
And, if you believe me or not: You’re very likely also using pointers in your code. Probably not directly, but as soon as you call a method, a function or a form with any parameter, you’re using pointers (except if you’re using the
VALUE()-syntax in the method header).
Calling methods with parameters means copying values. Each time a method is called, there is stack allocated for all parameters and local variables of that method, and the parameters are copied into that stack layer.
Unlike other programming languages, where the values passed are really copied, so the called method can use them as local variables, ABAP passes parameters by ref by default. That means, it doesn’t copy the whole value, but grabs a pointer in the background and copies only that few bytes to the new stack layer. When using the passed value in the method, ABAP then transparently dereferences the pointer to make the parameter seem like a real variable. I believe, that that behaviour was introduced to compensate that internal tables are treated like they are lying on the stack. By passing the values by ref, big internal tables don’t have to be copied, but just their pointer is copied which is much faster while remaining fully usable in the called method.
Using that knowledge, we’re able to do things like:
METHOD run. DATA(my_line_ref) = me->get_line( me->huge_table ). WRITE my_line_ref->*. "Works "The ref points to some line inside of "me->huge_table and therefore to the heap ENDMETHOD: " --> table TYPE TABLE OF i " <<< line_ref TYPE REF TO i METHOD get_line. READ TABLE table INDEX 1337 REFERENCE INTO line_ref. ENDMETHOD.
get_line, we’re passing a huge table. But as parameters are passed by ref, the table isn’t copied and just a pointer is passed to the method. That means,
table points to the same memory area
If we’re now reading from that table and save the reference of one line and return it, the caller can safely use that reference as it points to some line in table
It was afternoon and I was about to use the remaining time to fix some bugs in a freshly developed program.
While programming and testing, my program crashed with error
SYSTEM_DATA_ALREADY_FREE. That error occurs, when you try to dereference a pointer which points to freed/cleared/invalidated stack memory like in the first example. But I knew that I haven’t grabbed pointers to any stack variable as I’m paying attention to create all data, I want to share, directly inside the heap.
I followed the stacktrace and found a code with following structure as the origin of the error:
METHOD run. DATA(line_ref) = me->get_line( ). "Do something with line_ref ENDMETHOD. " <<< line_ref TYPE REF TO gtype_line METHOD get_line. line_ref = me->read_line( me->get_some_object( )->some_table ). ENDMETHOD. " --> table TYPE TABLE OF gtype_line " <<< line_ref TYPE REF TO gtype_line METHOD read_line. READ TABLE table INDEX 1337 REFERENCE INTO line_ref. ENDMETHOD.
As you see,
get_line only combines
read_line with an attribute of some object. The object is obtained by calling
me->get_some_object( ) and from that object, the attribute
some_table is accessed inline and passed as parameter to
read_line. As parameters are passed by ref, I don’t have to worry about the ref returned by
read_line as I passed a variable from heap and therefore the ref also points to the heap.
But the program crashes as soon as I’m trying to use
line_ref in the
run method. So I jumped into the debugger to look what’s going on there. It turned out, that the ref returned by
get_line wasn’t bound and pointed to freed memory. Although not a single stack variable was involved!
So I debugged the application step by step to eventually find out, that the ref becomes unbound/the memory behind the ref becomes freed, when the
read_line method returns.
But, that’s impossible!
read_line uses a variable from heap to grab a reference! Does the garbage collector free heap memory even if there are references?
Well, no. The garbage collector does its job right and hasn’t removed anything. But why does my line ref become invalid when the method returns?
I did some changes to my code and found the one thing that changes the behaviour of ABAP to no longer invalidating my pointer:
" <<< line_ref TYPE REF TO gtype_line METHOD get_line. DATA(some_object) = me->get_some_object( ). line_ref = me->read_line( some_object->some_table ). ENDMETHOD.
When storing the object ref in a local variable first, and accessing the attribute using that var, everything works as expected and my ref remains bound and usable.
But, why? Using a variable containing some value or using the return value of a method makes no difference! Both times, the return value of the method is copied to the stack and then used. It’s exactly the same value. In this case, it’s both times the reference to the object returned by
While experimenting with this issue (and revealing another weird behaviour of ABAP), it always did the same thing: While being in the method which reads from a passed table, the reference is valid. When returning from that method, the ref immediately points to freed memory.
Everything pointed to one cause: The table I’m reading from, _must_ lie in the stack. And it _must_ lie in the stack layer of the method reading from it (
read_line). But that would break the fundamental rule of ABAP method calling! Method parameters aren’t passed by val! They aren’t copied into the stack! And why does this behaviour only appear, when directly accessing attributes from returned objects inline, but not when accessing them through a variable?
In the meanwhile, a friend joined me to help searching the reason. I already left the office an hour ago, but I wanted to know the reasons and I already spent two hours on this little bug!
After testing another 30 minutes, we did the one test, that revealed, what’s happening:
METHOD get_line. DATA(some_object) = me->get_some_object( ). me->read_line( some_object->some_table ). me->read_line( me->get_some_object( )->some_table ). ENDMETHOD. " --> table TYPE TABLE OF gtype_line METHOD read_line. DATA(obj) = me->get_some_object( ). INSERT VALUE( ... ) INTO TABLE obj->some_table. ENDMETHOD.
While being in
read_line, we modify the table that was passed to us. But we don’t modify it through the parameter
table, but through the object obtained by
me->get_some_object( ). As
table was passed by ref, it points to the same memory area. That means, when modifying
obj->some_table, we should see the change also in
And indeed, the first method call in which
some_object->some_table was passed confirmed, that our test was working. The debugger showed that the table behind both variables
obj->some_table growed as the table was passed by ref and it’s one single table which is referenced by both variables.
Then, we debugged the second method call which uses
me->get_some_object( )->some_table and we waited for the debugger to again display the change of the table behind the both variables.
But, we got another result. While
table kept the state the table had before modifying it. And that means:
table MUST be a copy of the original table! ABAP must have copied the memory area of the table! That also explained, why the reference becomes invalid as soon as the method is left. As the table
table was copied, it’s now lying in the stack of the method, the pointer grabbed with
READ TABLE also points to the stack. To a stack area, which gets cleared when the method returns to its caller.
And together with the previous test results, we were able to derive the following rule:
If an attribute is passed to a method in whose access-chain the return value of a method is used, the content of that attribute is passed by val instead of by ref
And yes, that breaks the fundamental rule of ABAP method calling. Well, now you know _when_ a value is copied, but it’s still a completely unexpected behaviour.
But, wouldn’t that mean, every time the return-value of a method is passed inline (instead of an attribute of a return-value) this behaviour is shown? Because, when using return values inline, the rule stated above should also apply, shouldn’t it?
Well, yes and no. Probably ABAP copies that value, but return values are always copied to the callers stack, which has to be done as the method returning the value lost its stack area so the return value has to be stored somewhere else. That’s the reason why you have to use
RETURNING VALUE(...) in ABAP as it’s just not possible to internally return a pointer like for the
IMPORTING-parameters. Other programming languages don’t have such a
VALUE(...) syntax as they always copy both method parameters and return values. So, when using return-values directly as method parameters, it’s not noticed, whether the value is copied one time or two times. It only matters, if the method returns a pointer and that pointer is dereferenced directly. The pointer itself was the return value and therefore copied, but the data behind the pointer stayed the same. Even if the pointer was copied, it still pointed to the same memory area.
I haven’t any idea how to google that behaviour, therefore I’m asking you: Did you know that? Is that documented somewhere by SAP?
Below you’ll find the code I tested with so you can debug and reproduce the behaviour.
CLASS my_class DEFINITION. PUBLIC SECTION. DATA: table TYPE string_table, self TYPE REF TO my_class. METHODS: get_me RETURNING VALUE(obj1) TYPE REF TO my_class, get_line IMPORTING tab TYPE string_table RETURNING VALUE(ref1) TYPE REF TO string. ENDCLASS. CLASS my_class IMPLEMENTATION. " <<< obj1 TYPE REF TO my_class METHOD get_me. obj1 = me. ENDMETHOD. " --> tab TYPE string_table " <<< ref1 TYPE REF TO string METHOD get_line. "This line should modify both me->table and tab INSERT `new line` INTO TABLE me->table. READ TABLE tab INDEX 1 REFERENCE INTO ref1. ENDMETHOD. ENDCLASS. START-OF-SELECTION. DATA: ref1 TYPE REF TO string, obj1 TYPE REF TO my_class. CREATE OBJECT obj1. INSERT `line1` INTO TABLE obj1->table. ref1 = obj1->get_line( obj1->table ). "ref1 points to a string as expected "Yes, there is no sense in calling a method like 'get_me'. "it's just to have a method call in the access-chain to "trigger the behaviour ref1 = obj1->get_line( obj1->get_me( )->table ). "As soon as the return value of a method is used in the access-chain of "the parameter which is to be passed, its value will be copied to the stack. "ref1 is a freed reference and cannot be dereferenced anymore EXIT.
There is indeed something weird going out there.
I didn't understand why
behaves differently than:
What is ideology behind not using CHANGING/EXPORTING parameters?
ABAP isn't my first (and also not my favourite) language. I'm working a lot with other languages and I believe in the basic programming concept, that each function or method that's called gets their parameters and can use them like regulare local variables.
So if a function has to change something in my variables, it should, in my opinion, take a pointer as parameter (of course, as written, ABAP also uses pointers, but only in the background, so the user doesn't see it).
And regarding the EXPORTING-parmeters: If a function needs to "return" multiple values, in my opinion, it either should return an object containing these values, or if that's not possible, the design of the code should be changed so the function doesn't need to return more than one value.
Of course, I don't think, that code that uses CHANGING and EXPORTING is bad, and that's all just my opinion, but I don't like that language-feature.
It sounds like a little conservative approach to me.
Is it based on your Java experience?
I don't object this approach, but this way you give up on some of ABAP's abilities/strengths (internal tables would be another example).
And as written, it is possible to write any code without CHANGING and EXPORTING, and I think ABAP has not solved internal tables as well as in other languages, because they are treated as if they were on the stack, which they do not.
Impressive first blog post. I think you will get along well with people like Jacques Nomssi .
Jacques, have you seen this behavior before? Do you have additional input?
From my side, I have a small question. I hope it's not a stupid question. If I'm returning an object in a "RETURNING VALUE", is the object getting copied? Or in which cases am I really passing a reference to my object?
Objects are never copied in ABAP as you can't hold objects on the stack. You are forced to use TYPE REF TO so every time an object is "copied", it's just the pointer that's copied.
Copying _objects_ in ABAP is impossible, as far as I know. The only language I know which allows that, is C++.
So do I understand correctly that this situation only happens if we talk about basic data types like tables and integers?
Yes, everything that's not declared using TYPE REF TO.
But with all other types than table (ints, strings, char-sequences, ...) it isn't noticed as these types aren't modifiable and I've never seen a method which just grabs a pointer to a parameter and returns it.
So, this behaviour is basically only relevant for internal tables.
If I understand well your point (sorry I didn’t read everything), what you are experiencing is normal. Method results are not kept in memory if you don’t assign them to variables, i.e. they are not kept outside nested calls. You must use a reference to a persistent memory (*). I guess Horst Keller could easily point you to the part of the ABAP documentation which explains that.
EDIT : typo Inside chain->outside nested calls
UPDATE 1 : (*) If you want to keep a reference to a returned value by a method, this method must be changed to return a reference, and have the referenced value stored in a persistent memory, i.e. anything except a local variable defined in the method.
UPDATE 2 : I still can't find it in the ABAP documentation, but there's this blog post by Horst.
Yes, that's normal. But I use the result to pass it to the next method. And I use a reference to a persistent heap memory and ABAP still copies the data.
I updated my comment with a link to one Horst post which mentions this kind of issue.
I read the linked blog post. But it seems to contain only the behavior I described in the prologue. That references become invalid, when the stack layer they’re pointing to is cleared. That’s normal and expected.
My issue is about references which should point to the heap, but unexpectedly point to the stack.
Sorry now I have tested the last code snippet (which is the TL;DR of your post), and I understand the issue. I don't explain why it works like that.
I don't think it makes sense that *if* we use an auxiliar local variable, everything works fine, but if we try method chaining, it fails miserably. Sounds like inconsistent behavior to me that should be fixed.
Would be great to get Horst Keller 's input on this.
I believe its the same behaviour as in this recently posted blog:
Yeah, that behaviour could be linked to my issue somehow, although it addresses the lifetime of values inside a stack layer while in my case, it's about the lifetime of values across stack layers.
Hey Max Jäger, thanks for sharing this interesting view.
I guess it's a build-in safety mechanism to prevent a certain level of obfuscation due to 'excess' method-chaining 😉
A very obscure way of enforcing a ruthless Law of Demeter 😀
This is actually a documented, if incredibly confusing, behavior of Functions and Expressions as Actual Parameters:
Horst Keller - this behavior might warrant a more prominent entry in the abap documentation 🙂
Isn’t it a small wonder that it is documented at all?
Alas, it’s just too much stuff to bring it to any point.
We had a “traps and pitfalls” article in SAP Professional Journal around 20 years ago and a guideline book 10 years ago Now, should we write another book? Ah no, writing the ABAP reference (and its infrastructure) is quite enough ...
It wasn't my intention to imply the documentation is inadequate. To the contrary - time after time I am amazed how comprehensive the documentation is 🙂 I for one appreciate the tremendous amount of work You've put into it.
The issue I'm pointing at here is that, in this case, this little tidbit of information (with profound consequences!) is unintuitively placed in:
ABAP - Keyword Documentation → ABAP - Reference → Declarations → Typing → Checking Typing → Functions and Expressions as Actual Parameters
If it were my call, I'd probably put it in under:
ABAP - Keyword Documentation → ABAP - Reference → ABAP Syntax → ABAP Statements → Operands → Functions and Expressions in Operand Positions
Hmm, interesting idea, but there I wouldn't see it. That's just a list of positions.
In fact, the place where it is, is the right one, cause it is central and linked from all positions, where typing is mentioned. Maybe I overestimate the will to follow links.
But I describe the behavior additionally now here https://help.sap.com/http.svc/rc/abapdocu_752_index_htm/7.52/en-US/index.htm?file=abenformal_parameters_oview.htm . Well, another central position, but who looks there?
Thanks for digging for that documentation!
I still wonder if there's any sense behind that behaviour, or if it's just a technical limitation of ABAP.
Indeed Max, it's hard to find any sense in it.
So this means the following works:
But the following doesn't:
Just like Shai Sinai has mentioned.
So this is forcing us to create an auxiliary variable in between, not nice.
I'm still a little confused.
1. What does "pass by value" for object instance actually mean?
Is instance being cloned?
2. How is different from explicit RETURNING VALUE ("pass by value")?
Everytime a var is written using =, or a method is called which used VALUE() in the declaration, the value in the variable is copied.
But: In ABAP, you have never direct access to objects. Objects don’t ever lie on the stack or as attribute in your classes, as you can store objects only using TYPE REF TO. That means, if there’s a variable obj1 which has the type TYPE REF TO my_class, and you use a = to assign that variable, you copy the content of the variable. But the variable doesn’t contain the object, but just a reference. Therefore just the reference is copied.
In your code piece the method get_me does nothing else than copying its own reference me to the return variable and you as caller copy that reference again to new_obj. So obj1 and new_obj are two variables containing the same reference. So there’s still one single object to which now two references point.
And pass by value is nothing else than a copy of the variable content as for =. So if your new_obj would be passed by val, its value (the reference to an object) would be copied and the receiver would have the same value in its reference meaning they both point to one single object.
So, objects are never cloned in ABAP. Only data that can be declared _without_ TYPE REF TO can be copied.
So to answer your questions:
You've actually confirmed my assumptions.
(I just wanted to verify there isn't any other internal implementation I'm not aware of).
That's the reason I'm still confused:
How does "When binding a function, a calculation expression, a constructor expression, or a table expression, the parameters are always passed by value, even if the formal parameter is defined as to be passed by reference." explain the behaviour of:
Shai Sinai :
is effectively a shorthand for:
Tables are NOT objects in ABAP (sadly).
I didn’t think this is how it’s interrupted (and I don’t see any reason behind it).
As I’ve said before, I thought this is shortcut of:
Can you confirm it?
Michał Lipnicki ,
Due to internal tables sharing mechanism (Sharing is valid until the object is accessed to be changed), your example actually works.
i.e. It seems it isn’t the case.
This was just an example to illustrate the mechanism - the lifetime of "this_table_is_a_copy" is limited to the parameter binding.
The purpose of the snippet is to illustrate, that you're actually passing a copy of the original table and the reference you get points to a line of that copy.
Should have written something along the lines of:
Also that's not how table sharing works - it's basically SAP's implementation of copy-on-write.
Thanks for the clarification.
I know this was only an illustration.
However, I'm not sure this is the case.
If it was, I would expect that the result of:
would be the same,
but it isn't.