Technical Articles

Frederik Hudak

January 22, 2021 1 minute read

Making Failure Tolerable with ABAP Unit

Right after you learn to include helpful messages in your test failures, you will immediately encounter the problem of failure messages which are incredibly long.

If only the test runner wasn’t so one-dimensional!

Luckily, while procrastinating on fixing a failing unit test, I found out how to get it to display multiline output.

To achieve this, we need the fail( ) method which has an extra detail parameter, which will work with multiline strings. \n line endings are needed in the detail string, because \r ends up being displayed as # in the test runner.

If we combine this with the quit-no parameter, we can output multiple failures in a row to add as much diagnostic information about the failure as needed.

And if you still can’t figure out the problem, just add

level = if_aunit_consants=>severity-low to your assertion.

The error is still there, but it is now tolerable.

Example code

(uses this test wrapper class)

REPORT zfh.

CLASS lcl_test DEFINITION FINAL FOR TESTING
  INHERITING FROM zcl_abap_unit_wrapper
  DURATION SHORT
  RISK LEVEL HARMLESS.

  PRIVATE SECTION.

    "! Method under test
    METHODS is_latin
      IMPORTING text          TYPE string
      RETURNING VALUE(result) TYPE abap_bool.

    METHODS:
      test_latin_knowledge FOR TESTING RAISING cx_static_check.

ENDCLASS.


CLASS lcl_test IMPLEMENTATION.

  METHOD test_latin_knowledge.

    DATA(text) =
|Lorem ipsum dolor sit amet, consectetur adipiscing elit. \r\n| &&
|quam. Vivamus rutrum accumsan quam id rhoncus. \r\n| &&
|Etiam blandit metus sit amet vestibulum hendrerit. \r\n| &&
|Aenean placerat auctor fringilla. Pellentesque iaculis tortor a urna facilisis, \r\n|.

    assert_false(
      act = is_latin( text )
      msg = `Latin test failed`
      quit = quit-no
      level = if_aunit_constants=>severity-low ).

    REPLACE ALL OCCURRENCES OF |\r| IN text WITH ||. " test runner outputs \r as #
    fail( msg = `Failure. Lorem ipsum is not latin.` detail = text quit = quit-no ).

    REPLACE ALL OCCURRENCES OF |lorem ipsum| IN text WITH || IGNORING CASE.
    fail( msg = `This should fix it:` detail = text quit = quit-no ).

    assert_false(
      act = is_latin( text )
      msg = `Latin test failed again.`
      quit = quit-no ).

  ENDMETHOD.

  METHOD is_latin.
    result = xsdbool( text CS `lorem ipsum` ).
  ENDMETHOD.

ENDCLASS.

4 Comments

You must be Logged on to comment or reply to a post.

Sandra Rossi

January 23, 2021 at 10:04 pm

What I used to know about a tolerable exception is that it’s set via the parameter LEVEL = IF_AUNIT_CONSTANTS=>TOLERABLE. Note that a tolerable exception should indicate a “warning” yellow icon, not a failure blue icon. I’m not clear about the relationship between tolerable and level = if_aunit_constants=>severity-low.

I’m much more interested by the other part of your blog post concerning the output of several lines via FAIL( msg = ‘…’ detail = |line1\nline2| quit = if_aunit_constants=>quit-no ).

I wonder whether this feature can be used to create a helper class to:

log a complete “Given When Then” text when the test fails
alternate comparison algorithm because the standard algorithm sometimes doesn’t show clearly what is different and requires to debug to use the debugger comparator, especially for long strings, structures and internal tables)

For instance, I think of:

METHOD test.
  DATA(sbe) = NEW zcl_aunit_sbe( ). " Specification By Example

  sbe->given( 'Coffee machine has 10 doses left' ).
  cut = NEW zcl_coffee_machine( 10 ).

  sbe->when( 'I ask for a coffee' ).
  cut->ask_for_a_coffee( ).

  sbe->then( label = 'Coffee left' act = cut->coffee_left )->should_be_equal_to( exp = 9 ).
ENDMETHOD.

If coffee_left is not decremented, that should output:

"v Critical Assertion Error ”Equals”
  "> Details
    "> Expected 9
    "> Actual 10
  "> Stack
"v Critical Assertion Error ”Given the coffee machine has 10 doses left”
  "v Details
    "> 'Given the coffee machine has 10 doses left'
  "> Stack
"v Critical Assertion Error ”When I ask for a coffee”
  "v Details
    "> 'When I ask for a coffee'
  "> Stack
"v Critical Assertion Error ”Then coffee left should be equal to 9”
  "v Details
    "> 'Then coffee left should be equal to 9'
    "> `but it's equal to 10`
  "> Stack

I’m not aware of such a helper class. Your blog post motivates me to propose one. Thank you!

Frederik Hudak

Blog Post Author

January 25, 2021 at 8:17 am

Looking at if_aunit_constants, there is an assortment of constants in various states of deprecation. severity-low is actually the same thing as tolerable.

The assert methods usually output some kind of info about the differences, but I think it's not always useful and a general-purpose comparison could be done. You can serialize both objects to json, do a diff of the two strings and log only the changed parts as failures. In fact, you've already started!

Now if someone were to discover how to get colored text in there, that would really make my day.

Sandra Rossi

January 25, 2021 at 8:30 am

Oops, sorry about severity-low = tolerable (in fact I didn't pay attention that you were using LEVEL parameter; something strange is that you get a blue icon instead of yellow icon in your screenshot).

I am pessimistic about the colors, but I guess the DIFF result could be rendered not so badly by using some "text indicators" 😉

Sandra Rossi

August 10, 2021 at 12:49 pm

I just realize that I can't make the multiline feature work anymore on my (old) Eclipse 2020-09.

A standard workaround is to use CL_ABAP_UNIT_ASSERT=>ASSERT_THAT, it's then possible to return a table of String.

Technical Articles

Making Failure Tolerable with ABAP Unit

Example code

(uses this test wrapper class)

Assigned Tags