Skip to Content
Technical Articles

Cheaper Unit Tests with less Mock Logic

Automated tests have become an important part in my professional life as a software developer. However, you cannot always avoid external dependencies of individual methods you want to test. The more dependencies you have, the more cumbersome testing gets.

You may also read Start writing a Unit Test for examples how I make such tests.

Let me tell how I changed my test design over time to handle this. I’d really love to hear your opinion in a comment !

Different Approaches to automated testing

The standard advice is to mock any logic that is called by the code under test.

So, suppose tests are needed for such an application:

When each class is tested separately, mock logic has to be written for all coding the class depends on. This leads to implementing a lot of test coding:

The tests in this design typically refer to behavior that is invisible to the user, but “under the hood” (how a class interferes with other classes).

And even though the simplified diagram above does not show any boxes for the test coding itself, we already have a lot of boxes. Tests in this design may be difficult to understand. On the other hand, one of the tasks of these tests is exactly to enable a better understanding when you later get back to the code under test for some modification.

So why not write a single unit test?

There is much less test coding needed. The tests may also be easier to read as it is likely that they are more related to the original user requirements. I used therefore green color for the test coding.

Yes, in this simple example, it looks like an integration test. Especially in more complex software, it does make sense to cover a broader area of the functionality with automated integration tests, rather than focusing too much on a single part of the application. You will end up getting tests for the “right size”of code units (relative to your needs), instead of being forced into having isolated local test classes with every production code class.

Version 1 of my test designs

Already many years ago, the Clean Code initiative by Robert C. Martin (Uncle Bob) tried to minimize code complexity. The ABAP programming guideline by SAP contains much helpful advice, too, but testing was hardly covered by the original 2009 edition. I had the impression that Clean Code was more popular than automatic testing.

Now my daily job deals with ABAP in BW systems, sometimes also SAP BW transformations, Planning functions and other high level “coding”.

I wrote my first automated test in 2013, for just one complicated function to count days for an HR reporting application. I could do the test, because this function had near to no dependencies to other coding. The rest of the application remained without automated tests, because I did not know how to deal with dependencies.

Even when I learned to handle external dependencies in tests a few years later, my test coverage stagnated at about 5% of the code, because adding mock logic (either by inheritance or by interfaces) was too much effort to me.

Version 2 of my test designs

This changed when I was able to use ABAP Test Seams starting 2016. Test Seams are special statements of SAP ABAP to handle dependencies during automatic tests. They have restrictions, but the good point is, that they are easy to use. So with Test Seams it was easy to add automatic tests, even when dependencies made testing difficult. This changed the situation for me completely. Before that , I wrote automatic tests rather rarely after I decided that I really needed them. With Test Seams there was no excuse anymore. My new motto was: “Bad tests are better than no tests”.

I greatly increased my test coverage. When I needed to adapt existing code, I often added tests. With Test Seams, this is less risky and cheaper than with conventional techniques. Test seams also deal very well with legacy ABAP code that does not contain interfaces or classes that can be mocked easily.

Version 3 of my test designs

In 2018, I read this tweet…

Ian Cooper speaks about his experiences with automatic tests and all the errors he made. It is an excellent presentation, have a look for yourselves !

He advised to read the original book by Kent Beck about Test Driven Design. In his concept, the idea about unit tests, is not testing individual classes, but about the proper size of code to be tested. Not too small and not too big. That may be an individual class, but it may also be a differently sized unit.

So after seeing this presentation, I changed may testing style once more.

I tried to mock only when I found it really necessary. In all other cases I read and wrote from the database (in the development system). Sometimes I was able to keep the test data separate from other data. But often this was not possible. So I did it nonetheless, accepting the risk that a test breaks because someone changed the data. Actually, that occurred sometimes, but not on a regular basis. As a benefit, I needed less mock coding and less test seams.

Ian Cooper also advised that Test Driven Design means:

  1. Add or adapt a test so that the tests break.
  2. Write a code that fixes the broken tests. But do not write it beautifully, write it fast.
  3. When the tests pass: Refactor and think about improving the code you wrote.

With this in mind, writing automated tests felt even more simple. In many cases, I needed no mock logic at all.

Of course, testing still means some effort. For each new application you have to think how it can be tested well.

  • Having a separate “space” in the data reserved for testing is my preferred way.
  • Sometimes (not very often), Mock coding is needed.
  • Sometimes adding a Test Seam is the fastest and best method. The amount of tests seams has decreased significantly, however, as I avoid mock code as far as I can.
  • As the tests are not related to a single class anymore, I often implement special classes just for the unit tests.

Currently I focus on learning how to write more readable test code. I include a lot of helper methods just to have easy to read tests. All distracting statements are refactored into separate methods.

For an example, please have a look at ABAP-Unit-Test-Demo on Github. It is an example for a test where reading from the database is not mocked. The former test design looks like this (get_data and get_data2 are test methods):

  METHOD get_data.

    f_cut = NEW #( ).

    DELETE FROM zunitdemo_table1 WHERE test = 'X'.
    DATA: new_data TYPE STANDARD TABLE OF zunitdemo_table1 WITH DEFAULT KEY.
    new_data = VALUE #( ( test = |X|
                          key_a = |A|
                          field_1 = |C| ) ).

    INSERT zunitdemo_table1 FROM TABLE new_data.
    COMMIT WORK. " Required in case the data shall really be stored or deleted

    DATA: read_data_act TYPE STANDARD TABLE OF zunitdemo_table1 WITH DEFAULT KEY,
          read_data_exp TYPE STANDARD TABLE OF zunitdemo_table1 WITH DEFAULT KEY.

    read_data_act = f_cut->get_data( ).

    read_data_exp = VALUE #( (
                          mandt = sy-mandt
                          test = |X|
                          key_a = |A|
                          field_1 = |C| ) ).

   cl_abap_unit_assert=>assert_equals( msg = 'Expect correct data' exp = read_data_exp act = read_data_act ).

  ENDMETHOD.

The simplified test design looks like this:

  METHOD get_data2.

    " Version of test method get_data which is easier to read

    _prepare_test_data( key = |A| field = |C| ).

    read_data2_exp = VALUE #( ( mandt = sy-mandt
                                test = |X|
                                key_a = |A|
                                field_1 = |C| ) ).

    _test( message = |Expect correct data| ).

  ENDMETHOD.

The problem becomes very obvious when a test class has about a dozen of such tests. It becomes difficult to read and understand what exactly is being tested there. A couple of tests like in get_data2 are much easier to read.

Yes, the second test uses global variables, but I don’t think this is a real problem.

What do you think? What are your experiences with automatic tests? Feel free to comment below.

Rainer

My special thanks go to Thomas Keller and Benedikt Herrmann for valuable feedback and improvements of this blog!

 

 

 

 

 

 

23 Comments
You must be Logged on to comment or reply to a post.
  • Great blog! OK, biased opinion since it’s one of my favourite topics 🙂

    The main issue with the application in your first diagram is cyclic dependencies, one of the biggest enemies of unit testing. So the answer is really not how to test it, but how can you make it testable.

    Dependencies should only flow one way across components and never loop back, then there should be little or no redundancy in the tests and only a single mock should be needed for each component. So if class A calls class B which calls class C, then each only needs a single test and mock for each component. More often than not, the majority of work in adding tests to legacy code is actually modifying the code itself to make it testable.

    Personally, I’ve never used test seams. Usually when code needs test seams, tests tend to add little value in my opinion. I used to go with ‘bad tests are better than no tests’ too, but that theory has come back to bite me several times. Nowadays I rather add some kind of abstraction and try to do it properly straight away or not at all.

  • First of all I 100% agree with higher level tests. Even Robert Martin said something similar – i.e. avoid having test code that mimics all the private methods of your classes.

    Instead a have a bunch of unit tests in the “application” class that calls (say) the model (real) the controller (real) and the view (mocked).. This does indeed mimic some business requirements which talk about the user entering data (faked in the fake view) and then the rest of the program dealing with it.

    Second, if you use the “dependency lookup” technique and always get class instances via factory methods,  you then inject the test doubles during the SETUP method and nothing needs to be complicated at all. Every single one of the classes working together will get the same fake test double when they call the factory method.

    Third – in regard to unit tests writing and reading form the database. I am never going to agree to that. For a start unit test are supposed to run lightning fast and as soon as you start having database reads and writes and COMMIT WORKS that speed goes out of the window. In higher ABAP releases there are mechanisms to fake database access for both ABAP SQL and CDS views.

    Most of the database reads I do in my database access classes for which I create test doubles are for standard SAP tables like VBAK/VBAP and T001W and other configuration tables. I don’t think it would be a good idea for my test classes to blank out VBAK in development or put fake data in the configuration tables. I would say the same holds for the Z tables as well.

    Lastly – TEST SEAMS. Just no. I wish they had never been invented. They are an ABOMINATION. The production code knows it is being tested, so it is just like writing “IF IF_UNIT_TEST = ABAP_TRUE THEN” and so on. I literally do not see the difference. The analogy I always use is Volkswagen setting their vehicle emissions to do one thing when being tested and another in real life (which is what they actually did). If the production code does not know it is being tested such things are a lot harder to pull off.

    As far as I can see the effort/risk in adding a TEST SEAM to productive code is exactly equivalent to replacing the database call or authority check (or whatever) with a call to a method of a specialised class which can be mocked at unit test time.

    Finally I agree with making test methods easy to read. I never have any statements in test methods just calls to other methods as in:-

    METHOD test_method.

    given_something( ).

    when_something( ).

    then_something( ).

    ENDMETHOD.

    That way it looks like the test script. the test method says WHAT is happening, The lower level methods are more to do with HOW it happens.

    As you will have guessed this is a very emotive subject, so please feel free to argue with me all you like.

    What I will say is that I am really happy that the argument on SCN has moved away from whether to use unit testing or not, and now focuses on how to use unit testing.

    Cheersy Cheers

    Paul

     

     

     

    • I’d use test seams with one scenario only. To write unit tests for legacy code with minimal changes to the legacy code as the first step of getting the code into a test harness and refactoring.

      • To be honest I don’t see the difference between replacing something like

        SELECT * FROM VBAK WHERE VBELN = ‘THING’ INTO HEADER_DETAILS.

        with

        HEADER_DETAILS = DATABASE_READER->GET_HEADER_DETAILS( ‘THING’ ).

        and

        BEGIN TEST-SEAM order_details.

        SELECT * FROM VBAK WHERE VBELN = ‘THING’ INTO HEADER_DETAILS.

        END-TEST-SEAM.

        In both cases you have to change the legacy code, so it is an equal risk either way, given that any change to code is a risk.

        The worst thing to me is that you could put a TEST-SEAM around business logic if you desired.

        • I see your point, but i’d contend in this scenario, test seams would be the safer approach. Harder to screw up. And yes, I have screwed up even a simple replacement like that!

  • I’ve never had to write unit tests for B that needed a mock of A. I’ve never designed an application where B even knows A exists.

    Doesn’t the dependency you’ve shown indicate that you’ve got tight coupling that needs to be decoupled?

    • You are probably right. I did not really think about the details of the first diagram. I just needed an example for my blog.

  • Hi Matthew Billingham, Hi Paul Hardy, Hi  Mike Pokraka,

    thank you for your statements regarding test seams. I think currently about writing a further blog just about the question when and where to use test seams.

    I use currently test seams when I have the feeling that one of these points is significant:

    • The test seam reduces the cognitive load
    • The test seam reduces the code complexity
    • The test seam allows me to use a simpler architecture or programming methodology, this may be appropriate or even required for some customers
    • The test is restricted to a single class or function
    • Performance is critical: Method calls in ABAP require more time than method calls in other computer languages. Without a test seam I might need a method call which may be too time consuming for some applications
    • When the benefits of other test patterns appear to me to be not worth the additional complexity and effort

    Please give me time to reply to your points in detail in a separate blog.

    Cheers, Rainer

     

    • Dear Graham,

      thank you for your positive feedback!

      But please allow me to reply to your final thought, as it will take some time before I am able to write a new blog focusing on Test Seams.

      Test Seams are an additional option to implement tests. They are for me just a language statement.

      To call a language statement an abomination is for me only justified when it is proved that there is no use case where it is helpful.

      I see also a difference between using a language statement the first time. And using after having learned what can be done wrong. Regarding Test Seams I propose to follow a guideline when they are used. I posted mine here ABAP Test Seam for Unit Test with external dependencies – Personal guideline.

      I know of arguments against Test Seams. But arguments are for me not enough to prove that this statement should not be used. The arguments I know are given in blogs or books. But I am not aware of scientific reports which report the results of detailed evaluations.

      I assume that such a work will be costly and time consuming, and I have currently no funding to do it.

      Cheers

      Rainer

      Paul Hardy Matthew Billingham Mike Pokraka

       

      • I believe test seams was something that seemed like a good idea at first but once implemented (in the ABAP language that is), reality set in and it turned out to be a Bad Idea.

        I never saw the advantage other than saving 10-30 minutes over just implementing a class that can be reused for “proper” unit tests later on. The technical debt that remains almost certainly will be more than the little bit of time saved by using test seams as a shortcut.

        While I find your workarounds, patterns and safeguards for using test seams laudable, I still think the time could be better invested in remodelling into a proper abstraction class. For example, a good principle you write about in your other blog is about ensuring all seams are injected, but what about when it comes to refactoring the legacy code… you will need to change that all over again.

        To me this is so clear I don’t see the point of a scientific study. It’s not that much effort to whip up something along the lines of (partial pseudo-code):

        "Before:
        SELECT * 
          FROM dbtab 
          WHERE f1 = foo AND 
                f2 = bar 
          INTO TABLE @data( itab ). 
        
        "After:
        data(dbtab) = lcl_dbtab=>get_instance( ).
        data(itab) = dbtab->select_where( f1 = foo f2 = bar ). 
        
        "Local classes:
        INTERFACE lif_dbtab. 
          METHODS select_where.
        ENDINTERFACE. 
        
        CLASS lcl_dbtab DEFINITION.
          PUBLIC SECTION. 
            CLASS-METHODS get_instance.
            CLASS-METHODS set_instance.
        ENDCLASS. 
        
        CLASS ltc_test IMPLEMENTATION FOR TESTING. 
          METHOD SETUP.
            lcl_dbtab=>set_instance( NEW lcl_mock( ) ). 
          ENDMETHOD. 
        
          METHOD run_test. 
            ...
        

        Yes it’s a few more lines than a test seam, but having some code patterns handy makes it fairly quick, and its extensible and is not temporary code but the first step towards refactoring the product code.

      • To call a language statement an abomination is for me only justified when it is proved that there is no use case where it is helpful.

        One word.

        GOTO

         

        (In fact I’ve been designing a syntax for the rules for a rule engine, and when I was suggested using GOTO, you’d have thought I’d suggested publishing all the spoilers for the second series of Green Eggs and Ham on Netflix).

  • In regard to

    The test is restricted to a single class or function

    You cannot use TEST SEAMS in OO code (i.e. methods of classes) you get a syntax error and good thing to. They were only ever intended as a “bridge” for procedural code.

    Going back to my comment about it is good we have moved on to talking about HOW to use unit testing…

    What is not a good thing is that 20 YEARS after the ability to do OO programming in ABAP was introduced everyone is still saying “Oh I need to use TEST-SEAMS because 100% of my custom programs are procedural!”

    Cheersy Cheers

    Paul

    • Hi Paul,

      I agree Test Seams are no silver bullet. And the decision to use Test Seams makes it difficult to split a class.

      Cheers

      Rainer

  • Hi Rainer, I do unit tests for each class where I am able to create unit tests. I think that integration tests (testing more classes at once) and unit tests of individual classes are complementary, not excluding each other. Integration test should test different things as unit test, they should not overlap too much. When I started with unit tests then I also created few unit tests which were accessing database (only for ztable) like you did. It is really tempting to create unit test going to the database because you can better test your sql statements. But I stopped doing this because of reason mentioned by Paul, that speed of unit tests will suffer , and also because of the reason you have mentioned– unit test is then brittle – it can fail because of some bad connection to database or that someone would change your data in the table.

    What I am not sure, I am curious,  is that if a test class is created to do an integration test for more classes, then how is possible to find such a class if somebody other would create it in my team. So if I would have such a test class for class A,B,C and I would update class B, and break something in class B, then how I would know that there exists a test class which has now failing test case?

    I am on release 7.4 but I would not use test seams in new OO code what I would create because using dependency injection seems to be more clear for me, and the independent class (dependency) with interface I could reuse in the same project for another class. But Paul mentioned that there would be syntax error if test seams is used in OOP.

    For Selecting from database the OSQL test framework looks nice (currently, for me, I did not try it, so I am not sure what I would say if I would start to use OSQL).

    Have a nice day.