How to do convenient multicase unit tests with zmockup_loader
Some time ago I published a post about a mockup loading tool we use for unit testing in our projects http://scn.sap.com/community/abap/blog/2015/11/12/unit-testing-mockup-loader-for-abap. Since then me and my team were using it intensively in our developments and also introduced a couple of new features. This post describes our apprach to unit test preparation.
Just briefly for those who didn’t read the previous post: the idea of the mockup loader is that you prepare your test data in excel, copy it to a bunch of text files (manually or with excel2txt script that is also availble in repo), zip them, upload as binary object via SMW0, and use mockup loader to extract and deploy this data dynamically for your unit test. For more details see the previous post (link above).
Use case
Let’s suppose we have a method which is designed to identify quality of some product. We input a table of quality inspection parameters and expect some quality class assessment, in the example “A” or “B”. Certain parameters are compulsory and must not be skipped during quality inspection: if they are missing the method raises an exception.
The goal: write a unit test code once and then easily update it with new test cases prepared in Excel.
Data preparation
Let’s say we prepared a unit test table with 3 cases (docno = 10, 20 and 30). Note that QUANTITY parameter is missing in the document 30 – exception expected.
DOCNO | PARAM | VALUE |
---|---|---|
10 | WEIGHT | 12,89 |
10 | QUANTITY | 99 |
10 | MOISTURE | 12,6 |
20 | WEIGHT | 14,89 |
20 | QUANTITY | 103 |
20 | MOISTURE | 11,9 |
30 | WEIGHT | 14,89 |
30 | QUANTITY | |
30 | MOISTURE | 12,10 |
Another table is the index of test cases. Each refers to a DOCNO, describes expected result and describes the case in a human-readable form (as a kind of documentation and for failure messages). TYPE field describes if the case is positive or negative.
TESTID | TYPE | DOCNO | EXPECT | MSG |
---|---|---|---|---|
1 | + | 10 | A | Case with quality A |
2 | + | 20 | B | Case with quality B |
3 | – | 30 | Case with a missing Param |
Unit test code
1) Define the test case index structure (in the test class)
class lcl_test definition for testing inheriting from cl_aunit_assert
duration short risk level harmless.
public section.
types:
begin of ty_test_case,
testid type i,
type type char2,
docno type char10,
expect type char1,
msg type string,
end of ty_test_case.
data at_cases type table of ty_test_case. " << index table
data o_ml type ref to zcl_mockup_loader. " << mockup loader instance
data o type ref to zcl_class_under_test. " << class under test
...
2) In setup() method acquire a mockup loader instance and load index table
method setup.
data lo_ex type ref to zcx_mockup_loader_error.
create object o importing ... " << Object under test initialization
try.
o_ml = zcl_mockup_loader=>get_instance( ).
o_ml->load_data(
exporting i_obj = 'TEST1/index' " << file path inside zipfile
importing e_container = me->at_cases ).
catch cx_static_check into lo_ex.
fail( lo_ex->get_text( ) ).
endtry.
endmethod.
3) And here is main test cycle. BTW one of new features of ML introduced recently is data filtering – see the call of o_ml->load_data() – it passes the field name and value. If specified, the mockup loader filters the output to meed this condition. It is possible to define more complex filtering also (more details @github homepage).
method test_assess_quality.
data l_case type ty_test_case.
data lo_ex type cx_root.
data lt_insp type zcl_class_under_test=>ty_inspections.
data l_act type zcl_class_under_test=>ty_quality.
loop at at_cases into l_case. " << Loop through cases
clear lo_ex.
try.
o_ml->load_data(
exporting i_obj = 'TEST1/inspections'
i_where = 'DOCNO = ' && l_case-testid
importing e_container = lt_insp ). " ^^^ Filter only relevant lines
l_act = o->assess_quality( lt_insp ). " << Test call
catch zcx_mockup_loader_error into lo_ex.
fail( lo_ex->get_text( ) ). " Mockup load error handling, just in case
catch zcx_root into lo_ex.
" do nothing - this is the expected exception for negative tests
" better use specific exeption classes of course
endtry.
if l_case-type = '+'. " Positive test
" msg indicates the failed test id and it's description
assert_initial( act = lo_ex
msg = |[{ l_case-testid }] { l_case-msg }| ).
assert_equals( act = l_act
exp = l_case-expect
msg = |[{ l_case-testid }] { l_case-msg }| ).
else. " '-' " Negative test
assert_not_initial( act = lo_ex
msg = |[{ l_case-testid }] { l_case-msg }| ).
" Potentially more specific error check code should follow
endif.
endloop.
endmethod.
4) That’s it !
Now we’ve got an easy extendable test infrustructure. New test cases for this test can be added without extra coding. Honestly speaking, this works better for integrated tests than for checking small methods. However, it is very useful when applicable – some methods in out developments have unit tests with 20+ cases and they are still updated from time to time with new ones (usually after yet another bug discovery 😉 ).
Hope you find it useful ! 🙂
The ABAP mockup loader project lives @github, there you can find full reference of the methods and also some use cases in wiki.
All the best, Alexander Tsybulsky.
How would you compare this method to using a mocking framework (either an existing one licke mockA or rolling your own)? I'm looking into simplifying test data setups and mock setups because it is a lot of manual work but not sure of the best approach yet.
I didn't use mockA in real product much but I had studied it a bit some time ago. My brief conclusion - they do not compete, in fact they do a bit different things and potentially supplement each other.
From what I learned, mockA is able to create instance objects for given interfaces. And fill it with predefined answers. And it does it's job in convenient and elegant way. Really. You must hard-code that however (afaik).
Mockup_loader's purpose is to load data. You prepare it in Excel, convert to text, load via SMW0 and then the test data travels with the package. We deploy our products to several clients so this is very relevant for my company. In one of our products we have 107 text files ( = different test data structures), ~250k of pure text, ~1500 of data lines. Would you imagine hard-coding this amount of data ? The mockup loader simplifies this a lot!
However after you loaded the data and it comes to feeding it to the code-under-test and if you use interfaces - mockA seems to be a very good extension to the flow. (I'm considering this actually but we don't have that many interfaces in our products).
In addition to described in the post, the mockup_loader has a store()/retrieve() mechanism which can be used to deliver the test data also (i'm going to write about it another post soon, but it is described at reference.md at github also). However, it is a bit more straight-forward approach and requires a call from code-under-test. Which is somewhat arguable conceptually (this leads to a very theoretical discussion though =)
Thanks for the detailed answer.
Yes you're right it looks like the two are not really solving the same problem. I haven't come across a situation yet where I needed to import large amounts of test data so they're mostly just hard coded in the tests but clearly your products are different.
Having the mockup loader referenced in the production code does seem a bit too much but hey. 🙂
> Having the mockup loader referenced in the production code does seem a bit too much
Well, if carefully used it is OK 🙂 I'll write another post on store()/retrieve() functionality. But briefly: in the product where we use it we have a singlton context which, among other, define if it is a test run. So the below construction appears to be safe enough:
Alexander, I really like this extension to enter mock data massively and user-friendly without ABAP knowledge! I hope it could be extended with a maintenance screen to edit directly the mock data to avoid the steps! Keep up the good work 🙂
Sandra, thanks for the feedback. There were no plans for that at the moment - I'd say that Excel is more convenient tool than any abap interface 🙂 But I'll consider it. Thanks ! I'm not sure if it is possible to display Excel interface in sap windows for editing ... if it is that it could be an interesting way indeed.
Yes it's possible (using Desktop Office Integration). But maybe it could be faster (from user perspective) and much more extensible to use an editable ALV to extend the input with other functions (I really think your tool could have many more interesting features).
Is it OK if I mention this in the next edition of my book?
As far as I am concerned this is the missing piece of the Unit Testing jigsaw puzzle, this can work hand in hand with the ABAP Test Double Framework for situations where you have hundreds of complicated inputs, each with an expected result, and you need to regression test the whole lot on a regular basis. I think I am right about this.
When I tell my users do this testing manually each time the program changes they usually objected, objected, objected, they usually objected.
And I'll allow, as you expect, that they are right to so object, and they are right, and I am right, and everything is quite correct.
My users are in love with Excel (like most people) and that is their preferred way of maintaining such data. I like the fact this gets uploaded to SAP, and at the very minimum I could write an ALV report to display the data.
In real life however my people would say that maintaining data in Excel is far, far, easier than maintaining that same data in an editable ALV grid. In fact they would argue that they want to use a spread sheet, a spread sheet, a spread sheet, they want to use a spread sheet,
And they are right , I think you'll say, to argue in this kind of way, and they are right, and I am right, and all is right, too loo rye ooh rye ay!
> Is it OK if I mention this in the next edition of my book?
Sure, that would be a pleasure 🙂
Indeed, the mockup_loader may co-exist and benefit from using along with other test frameworks. There was some comments about mockA in this respect above.
And I agree with you about Excel - Excel is fantastic 🙂