A Cheap Performance Evaluation

[nomssi]

When do we say that an application performance is good enough?

users will have no patience with an app that is not responsive but a performance problem in an ABAP report might only be revealed when it dumps with in production because of the sheer size of the data to be processed there. So how can we check if a process with an acceptable run-time in the development and quality systems will also have an acceptable response time for larger input?

Let us first note there what acceptable response time means. We might OK an one-time extraction process that runs for 10 hours in the background, or a report that takes a day to process a million entries in a data migration context, but veto an one-minute-wait for a single record access.

So the run-time alone is not a good performance indicator, we should focus on order-of-growth comparisons instead. But the analysis of algorithms can be quite complex or have some prerequisites many are not comfortable with. However, there is an empirical estimation method that is as easy as writing an ABAP unit test.

Some Theory

Caveat: We cannot avoid some maths here, but in those times of Covid19, I hope you might have seen animations explaining why exponential growth can be bad.

Let us assume we have a process that takes more time with a larger input (e.g. higher number of "rows" like in this concrete example of an ABAP problem).

From our vantage point (software), the running time T is described as a function T(N) of an input size N (e.g. number of "rows"). We model this dependency as power law

• T(N) = c N^b

where c is a constant.

What is the effect of doubling the size of the input on the running time?

Since the formula is relatively simple, the order-of-growth b can be easily estimated by comparing T(N) with T(2N) based on the doubling hypothesis, b is equal to

• log( T(2N) / T(N) ) / log( 2 ) .

If the order of growth b is 2 (quadratic) or higher, you have a potential disaster at hand if the input size cannot be kept small.

Some Code

I implemented this logic as a unit test. by using the fact that b is also equal to

• log( T(10 N) / T(N) ).

  METHOD profile.

    CLEAR rs_time.

    rs_time-size = iv_size.

    DATA(lv_start) = mi_timer->get_runtime( ).

    run_data( iv_size ).           " <---- The Code under Test

    rs_time-time = mi_timer->get_runtime( ) - lv_start.

    IF iv_time GT 0.

      rs_time-lg_ratio = log10( rs_time-time / iv_time ).

    ENDIF.

  ENDMETHOD.                    "profile



  METHOD performance.

*   empirical analysis

    CONSTANTS c_precision TYPE f VALUE '1e-2'.

    DATA lv_size TYPE syindex VALUE 1.

    mi_timer = cl_abap_runtime=>create_hr_timer( ).

    DATA(ls_time) = profile( iv_size = 0

                             iv_time = 0 ).

    DO 4 TIMES.

      ls_time = profile( iv_size = lv_size

                         iv_time = ls_time-time ).

      lv_size = lv_size * 10.

    ENDDO.

    IF ( ls_time-lg_ratio - c_precision )  > 1 .

      cl_abap_unit_assert=>fail( msg = | Performance: { ls_time-lg_ratio } | ).

    ENDIF.

  ENDMETHOD.

 

In this case, the input size are 1, 10, 100, 1000. The result is based on the comparison between the run-time for 100 and the run-time for 1000 "rows". So if you can measure your run-time as a function of the size of the input set, then you can already compute an order-of-growth estimation from 2 measurements.

 

Ref.: Introduction to Programming in Java by R. Sedgewick, K. Wayne

 

First published as https://www.informatikdv.de/a-cheap-performance-evaluation/