The mysteries of the RETURNING parameter
I really like the RETURNING parameter in ABAP OO, in my opinion it makes ABAP code more consistent and readable, see https://www.cqse.eu/en/blog/coding-abap-like-java/#keep-method-calls-simple-and-consistent for an example.
I use it whenever possible, also for deep structures and internal tables. However, the standard SAP code inspector check “Poor parameter pass performance” states that:
And RETURNING is always passed by value, which always gives a minimum of 40% performance loss!? And pass by value will use twice the memory?
Since memory is quite cheap, it might be worth taking the performance decrease of 40% in favor of simplified code? Let’s try and see what happens.
I’ve written a short test program, which calls 2 methods: passing by reference and passing by value. The code is available at https://github.com/larshp/return_by_value and the tests have been run on 750SP02.
With 1 million rows (around 1 GB) returned/exported 100 times, the averages are:
Using RETURNING does not seem to be 40% slower, but perhaps the base for the 40% is very small compared to the time required for populating the table, hmm
If pass-by-value copies the data, then the memory consumption needed for RETURNING should be twice that needed in EXPORTING. However, when looking at the memory consumption in the debugger the peak for RETURNING is the same as EXPORTING, but perhaps the actual kernel intermediate memory peak is not sent to the memory analysis tool.
Let’s try coping an internal table with 500.000 rows(500mb), and checking the memory usage,
lt_tab2 = lt_tab. BREAK-POINT.
2 identical tables with 500mb contents only takes 500mb, as it seems they share the same memory, i.e. only a shallow copy is made. Modifying the table contents:
READ TABLE lt_tab2 INDEX 1 ASSIGNING <ls_tab>. <ls_tab>-mandt = '123'. BREAK-POINT.
And each table occupies 500mb.
Internal Table Sharing
The concept of table sharing is described briefly in http://sapinsider.wispubs.com/Assets/Articles/2008/October/A-Developers-Guide-To-Protecting-Memory-Detect-And-Eliminate-Damaging-Memory-Leaks-With-ABAP-Memory which mentions “Table sharing also occurs when IMPORTING or EXPORTING parameters are passed by value”, but RETURNING is not mentioned.
Which provides a few additional hints.
As a special case table sharing is used for nested tables in RETURNING,
RETURNING can be used to pass large internal tables, if in doubt write a small test program to test the assumptions or try running the standard SAP code inspector check “Performance checks” -> “Poor parameter pass performance”. As everything else in ABAP, there are most likely some special cases which are not covered above.
Update: Also make sure to read the comments below, lots of valuable information
Very important post. Thank you Lars, now I have to change my training material again ?
Addendum: and if everytime I start a discussion on Twitter would lead to such a blog post, I'd do it more often....
It is always fun to discuss ABAP, there is always a new feature to discover. But usually 140 characters are not enough :o)
Nicely demonstrated and code made such a way one can check whether the result is the same on older systems, so thank you! For information, here's the same great result for a 7.31 system kernel 7.20 :
thanks, the article on SAPinsider is almost 10 years old, so I expect RETURNING to work without big performance loss on quite old versions.
I'm still surprised it works this way, always thought it copied the data, started investigating to figure out how big the performance loss was.
Thanks for clarifying that up.Until now i followed the letter of the law(read: CI) and avoided using big internal tables as RETURNING params!
Two things i’ll start doing after reading your blog:
Slightly off-topic; do you prefer EXPORTING/CHANGING params in conjunction with RETURNING params? Sometimes when i’m super lazy and don’t wanna refactor i add an EXPORTING param to method signature leaving the RETURNING param as-is.
Tbh, it doesn’t feel alright to have EXPORTING/CHANGING params in a functional method[y = f(x)]! What are your thoughts?
2. I think it is a good idea to keep the CI check, it has parts that are useful. The check description is a bit confusing, but I think the warning "RETURNING parameter contains internal tables" can be ignored, note that this is configurable in the check. Might investigate further sometime next week.
Mixing exporting and returning: it doesnt feel right, even though it is a new feature. Will probably add it as a check in CI some day, to make sure it is not used in a project, https://github.com/larshp/abapOpenChecks/issues/320
True that. I had already corrected my statement!
Under the "Memory Consumption" paragraph, you have stated
By VALUE() you mean pass-by-value params right? If i may suggest, could you explicitly as "pass-by-value"? I was thinking for a sec what has the VALUE operator to do with all this :-/
CI check: I've also elaborated in my previous comment.
VALUE(): thanks, blog updated
So if I understand it right. As long as the internal table remains unchanged, processing time and memory consumption remain the same with usage of exporting or returning?
But if I modify the received internal table after receiving it from the function, in case of returning a copy is still done.
Extract of the online documentation: Memory Inspector Concepts
Because internal tables and strings can become quite large, ABAP saves copying workload by employing a lazy copy strategy (Copy-On-Write). The initial sharing/revocation of initial sharing in static boxed components is analogous in its effect.
As long as the component remains unchanged, ABAP lets all variables that refer to the table or string point to a single memory object. Only when a table or string is changed via a variable does the ABAP runtime make a separate copy of the changed object. This lazy copy strategy means that changes to tables, strings, or boxed components can result in surprising jumps in memory consumption, as seen in the Memory Inspector.
Following the example from https://github.com/larshp/return_by_value
1: The method fills RT_VALUES
2: The method returns, this lazily copies the table to LT_TAB, note that this is fast
3: RT_VALUES is no longer in scope
4: only LT_TAB contains a reference to the table, so it can be changed without the kernel performing a slow copy
Note that it works in this example since the contents for RT_VALUES is generated in the method.
If class ZCL_RETURN_TEST stored the table in a member variable/attribute, and LT_TAB was changed in the report, then the kernel would have to copy the contents. Another example to test :o)
Yes - lazy copying is part and parcel of this whole thing.
I tried the same example with the storage in a member variable (private and public) and assigning it to the exporting and the returning parameter. The behaviour did not change significantly.
And if I changed the table lt_tab afterwards there was also no significant difference. But if I do not oversee something in the exporting and returning case there has to be made a copy in both cases, otherwise I could change the internal table of the class outside in the report.
Very valuable insights in this blog Lars, thanks.
added new example, see https://github.com/larshp/return_by_value/blob/master/src/zreturn_test2.prog.abap
Using RETURNING for passing large tables is (maybe) safe when:
A: Only returning locally scoped data
B: Not changing the returned table
However, if returning globally scoped and changing the data there is a performance hit as the kernel will have to copy the data.
Good "second conclusion", worth putting after the conclusion of the main post too 🙂
Thanks for your invest in checking this!
Need to deactivate the checks in SCI.
make sure to check this new example, https://github.com/larshp/return_by_value/blob/master/src/zreturn_test2.prog.abap there are some circumstances where performance can be impacted.
Still thinking out loud...
So we can reduce the problematic cases to GETTER methods returning structures and tables which you want to alter and write back with SETTER methods.
But I think, this also happens if you are using EXPORTING and "by ref". As soon as you change the exported variable in the caller object, the value will be copied (else you would change the original class attribute).
yeah, though not sure how it works with structures
Very interesting analysis! Here's some background on the principle for those who are interested: https://en.wikipedia.org/wiki/Copy-on-write
My approach to large tables is to use references: One class creates the table and returns or exports a REF TO the itab. Although a bit more fiddly to work with, this guarantees a single copy in memory. But with the knowledge that there is no impact if we know the data isn't being modified we can avoid de/referencing - always good to simplify code 🙂
As a general comment, using references has both an advantage and a disadvantage (depending on use case), in that a caller can modify data without the 'owner' being aware.
But this can be a good thing: e.g. if the original data 'owner' is a dumb DB access class serving up data to the application then this is exactly what we want.
If only we had a few more of the C++ possibilities, such as returning a const reference, because: "Rule: When passing an argument by reference, always use a const references unless you need to change the value of the argument"
The most benefit of returning parameter is to give ABAP the ability of method chaining, like other languages like Java and C++.
It may have some performance issue is because the returning value always pass by value, and always do the type conversion from the data type you declared and the data type of the variable you assigned.
There are two cases most useful for returning value
1.Use return value in abap statement, like if, string expression
In this case, the method usually return simple value
In this case the method usually return a object, and the value passed is the reference of the object.
So if you want to pass large and complex data object, there are no signifiant benefit of returning parameter and there may be some performance issue. ( It will happen when you modify internal table)
If you really want to pass big data object, the best way is to get the data reference of the variable ( using REF #( )) and pass the reference.
Good article Lars.
the convenience of "returning" , e.g, data(xxx) = new clas( )->methA( )->methB( )->methC( )....totally is worthy of the 40% performance loss 😉
worthy of 40%, but anyway the loss is 0% in your case