Application Development Blog Posts
Learn and share on deeper, cross technology development topics such as integration and connectivity, automation, cloud extensibility, developing at scale, and security.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

ABAP Trace to PlantUML Sequence Diagram

nomssi
Active Contributor
‎10-10-2022 12:40 AM

Synopsis


Some shortcomings of the SAP standard sequence diagram generation from ABAP run-time measurements are addressed:

  • a secondary internal table index speeds up parsing

  • a custom filter displays calls from standard to custom code

  • loop compaction produces smaller diagrams

  • the sequence diagram output in PlantUML Format is editable plain text .


Local installation of PlantUML is recommended but not mandatory as a web service is available. The JNet OCX is not needed.

Introduction



UML from ABAP Code


Transaction SAT profiles an ABAP program execution and provides analysis tools for performance, program flow and memory consumption. Measurements can be imported from other SAP systems using either download/upload or RFC. Since SAP Netweaver 7.02, an UML sequence diagram can be generated if the measurement was recorded without aggregation.

  • Start transaction SAT -> Evaluate -> Double Click on a measurement; on the next screen press F8 (Display measurement as UML sequence diagram).


A sequence diagram will be displayed if the JNet OCX component is installed (e.g. with SAP GUI). This feature was never fully delivered according to SAP OSS Note 1744063: development for this function has not yet finished and therefore contains some serious errors including program terminations.

Design-Time Sequence Diagrams


At design time, sequence diagrams are usually created manually before the code is implemented to visualize collaboration aspects of the desired system actors. Design time sequence diagrams accelerate the discussion on stable interfaces (e.g. compared with this SCN thread OO Analysis) before the system is built. Sandi Metz says that each message requires that the sender know things about the receiver. This knowledge creates dependencies between the two and these dependencies stand in the way of change

Let us compare two diagrams generated for reports from the thread Classical way to ABAP OO style of coding with the same functionality.



 

  • The first diagram reveals the logic: GET_DATA( ) selects some business data, and then DISPLAY( ) implements the output / user interface.

  • The second program attempts to separate concerns: the new actor LCL_MAIN exposes the business logic as a service requested by sending a START( ) message. The design pattern here is to evaluate which parts of the system are likely to change and to isolate those parts in new actors with carefully crafted interfaces.


With a stable interface, caller and called actors can change independently. As long as they abide by the contract, e.g. method START( ), there is no ripple effect of change through the system.

It is tempting to create layers of abstractions everywhere to increase flexibility, I made a more complex proposal in the use case above. The designer must ultimately decide if the increased complexity (new classes) is justified.

Motivation


Sequence Diagrams from Run-time Data


Our aim is to automate the sequence diagrams generation from run-time measurements. When the creation cost is low, the feedback loop is short, and we can routinely use sequence diagrams to appreciate the impact of code changes on the overall system design.

Object oriented ABAP code defines classes while the application behavior is better explained by messages exchanged between run-time objects. The appealing visualization revealing actual communication patterns makes it easier to discuss separation of concern, stable interfaces and simpler protocols for objects exposing too much of their inner structure.


Enhance Sequence Diagrams Output


SAP has provided an easy enhancement path for the UML class diagram output (subclass class CL_UML_CLASS_DECORATOR). Some hard coded limitations in the ATRA_UML_DECIDER function module and the CL_ATRA_FACTORY class must be overridden to enhance sequence diagrams by subclassing CL_ATRA_UML (implementation of the IF_ATRA_UML_TOOL interface). I have been using Chris Paine's code and I liked it very much. I now believe his breakthrough design decision was to delegate the graphical representation to an external tool. His code saves a sequence diagram in the format of the free PlantUML tool that requires Java on your machine. PlantUML syntax is human readable and editable plain text. A Chrome browser extension displays the output as soon as the text file is updated.

My first real contribution was to implement loop compaction, i.e., use UML combined fragments to express loops. This is done recursively until no further gain is achieved; it is gratifying to generate a small diagram with LOOP statements from large traces.



A future improvement could be to build a dictionary of non-adjacent blocks and increase trace compaction by using conditionals ALT or OPT combined fragments. I also found the trace parsing could be accelerated by using a secondary internal table index, so the current code replaces the standard sequence diagram display logic completely instead of extending it. I could consume the PlantUML web service in ABAP.

This leads to a short feedback loop and learning is enjoyable... if the image is not too large.

Avoiding Large Traces


Transaction SAT can generate very large traces, with much more than 1 million entries, that are impossible to process: the evaluation is extremely slow or fails with a timeout. This, I presume, is why the size limit is so prominently featured in the variant customizing. But it does not take the largest trace to generate diagrams too large to deliver useful insights into the design. Care is needed if we want to selectively trace program parts and so restrict the generated diagram to parts we are interested in. We can start a trace

  • In Dialog - the measurement variant can be setup to switch measurement off and on manually or with the statement SET RUN TIME ANALYZER ON/OFF, to discard some processing blocks and limit the measurement to given program components.

  • In a parallel session - the measurement is then always started/stopped manually

  • In a background session (for another user/service) - cf. Chris Paine's blog.


 

  • While evaluating an existing trace, we can use the filter dialog (Ctrl+F1) to narrow the processing to selected actors or messages. Chris Paine proposed a custom code filter that also displays calls from standard to custom code.

  • We can optimize the diagram generation for speed or size: Following advice from a PlantUML developer, I introduced a scale factor. Scale value 0.5 reduces the generated image size by 1/4.


 

Setup


Manual Steps


Class CL_ATRA_UML_HANDLING controls the generation of an UML sequence diagram from a measurement without aggregation in transaction SAT. Method SHOW_SEQ_DIAGRAM will call method DISPLAY_UML( ) of local class LCL_UML_HANDLING. Three steps are needed to override the standard behavior:

  • Create a new Include YY_SATRA_INCLUDE using the code available from github.

  • In class builder for class CL_ATRA_UML_HANDLING  Goto -> Local Definitions/Implementations -> Local Definitions/Implementations Ctrl+Shift+F6. The definition/implementation of local class LCL_UML_HANDLING is displayed. At the bottom of the source code, implement an implicit enhancement and insert the logic from the include:
    ENHANCEMENT 2 YY_SATRA_SEQUENCE. "active version
    
INCLUDE YY_SATRA_INCLUDE.
    
ENDENHANCEMENT.​


  • The last step is to override the call in method SHOW_SEQ_DIAGRAM( ) using an enhancement:
    ENHANCEMENT 1  YY_SATRA_SEQUENCE.    "active version
    
  
    
  DATA lx_error TYPE REF TO cx_dynamic_check.
    
  TRY.
    
      lcl_sequence=>to_diagram( lcl_configuration=>query( ) )->output( ).
    
    CATCH cx_dynamic_check INTO lx_error.
    
      MESSAGE lx_error TYPE 'I' DISPLAY LIKE 'E'.  "#EC CI_USE_WANTED
    
  ENDTRY.
    
  RETURN.
    
ENDENHANCEMENT.​


  • After activation, your system behavior will have changed. A new customizing popup appears while displaying a sequence diagram.



Customizing Sequence Diagram Generation



Trace Analysis Steps



  • Query ABAP execution trace created by transaction SAT without aggregation

  • Parse call hierarchy in trace, apply custom filters

  • Convert filtered trace to sequence diagram in text form

  • Save UML as text and generate and/or display image.


Domain Model


 



Classes LCL_ABAP_TRACE and LCL_TRACE, Interface LIF_COLLECTOR


Class LCL_ABAP_TRACE parses the call hierarchy (form routines, methods, function modules, screen flow...)  and actors (programs, function groups, classes, logical databases...) of an ABAP trace stored in static attributes of standard class CL_ATRA_TOOL_SE30_MAIN, i.e. the internal tables

  • IT_TRACEPROG  - Table of Recorded Program Names

  • IT_TRACETEXT - Table of TRACE Text Elements

  • IT_TRACEMETH - Table of Recorded Method Names

  • IT_AUSTAB_HIER – Call Hierarchy table with All TRACE Information


Class LCL_TRACE uses the logic inherited from LCL_ABAP_TRACE to transform trace entries into the target structure TS_SAT used for sequence diagram creation.





































Structure Field
Description
ID Indicator describes the type of the recorded event (call method, perform, call function, call screen, database operation, message...)
FROM_LEVEL Call level in the call hierarchy (an integer)
CALLER caller object - Source of the message (program, function group, class, etc.)
CALLED called object - Receiver (target) of the message
AUS_TABIX Index for table access
SYSTEM System flag " only used in custom filter


This structure is passed to any collector class like LCL_SEQUENCE implementing the LIF_COLLECTOR interface. A collector is responsible for applying custom filter. The COLLECT( ) method must returns ABAP_FALSE if the entry was rejected (filtered out) so the LCL_TRACE method can adjust the whole call hierarchy.  Some helper classes were introduced:



  • LCL_BAG - Abstract data type with the interface ADD/REMOVE/CONTAINS simplifies the filter logic

  • LCL_CLASS_NAME to transform between technical and external ABAP class name


Classes LCL_ACTORS, Interface LIF_ACTORS


The rest of the code uses the term Actor according to the sequence diagram definition to denote any object that sends or receives messages. The collection of actors (or actor dictionary) is accessed via a LIF_ACTORS interface that defines:

  • method NEW_PATH( ) saves the caller and called object from structure TS_SAT in the actor collection. New entries are created and existing entries are retrieved from the collection of actors. A unique key is returned for each actor, so a pair of actor indices (a PATH) is returned by the NEW_PATH( ) method for the caller and called objects.

  • the short text of a message (mostly determined by the called object/actor) is formatted using the SHORT_TEXT( ) method

  • method LIFELINES( ) returns the complete list (internal table) of sequence diagram lifelines. It is needed at initialization and completion of the sequence diagram.


Class LCL_ACTORS implements the LIF_ACTORS interface. The actors are stored in a sorted internal table MT_ACTOR with the unique secondary key OBJ_NR for performance. While comparing actors, a special treatment is needed for entries of type object (the object oriented instance of a class), since each object instance has its separate identity - cf method INDEX_OF( ).

Interface LIF_TRACE_FILTER, Classes LCL_FILTER, LCL_FILTER_NULL, LCL_FILTER_CUSTOM


The filter logic is abstracted to interface LIF_TRACE_FILTER to make it easy to try new ideas. The easiest way to create a new custom filter is to subclass class LCL_FILTER and redefine the method ACCEPTS( ) with a custom logic. Adjust the factory method NEW( ) of class LCL_FILTER to inject the new logic and no additional changes to the collector in LCL_SEQUENCE would be needed. We provide three filter implementations:

  • Class LCL_FILTER_NULL is a dummy filter without impact. It is only used if the Loops compacted checkbox is disabled and if no entries exist in the Customer namespace field

  • Class LCL_FILTER rejects some low-level object identifiers. It is used if the Loops compacted checkbox is disabled and the Customer namespace field is maintained.

  • The default implementation in class LCL_FILTER_CUSTOM applies a Customer namespace filter but also keeps actors from other name spaces calling that trigger messages to custom code (Chris Paine's logic).


Note: A lifeline without short text will be displayed if the actor is filtered out but the message is not.

Class LCL_MESSAGES


is an iterator pattern implementation that traverses the collected messages for UML sequence diagram generation. The canonical methods are

  • HAS_NEXT( ) - check for availability of an entry and

  • NEXT( ) - get the next entry for an internal table.


We further expose methods

  • SKIP( ) - skip a number of entries (default 1) 

  • IS_FIRST( ),

  • FIRST_LEVEL( ) and NEXT_LEVEL( ) to return the current call depth.


Classes LCL_STACK, LCL_UML_STACK


implements the stack (last in-first out queue) methods PUSH( ) and POP( ) to store call levels (structure TS_CALL_LEVEL with fields ACTOR_KEY and FROM_LEVEL).This implementation always updates flag MV_EMPTY but it does never throw any exception; instead, it returns an initial value when the stack is empty.

The subclass LCL_UML_STACK uses the stack to implement a call stack with methods CALL( ) and the RETURN( ) behavior depending on the call level. Note the LCL_STACK was only extracted to try a different implementation of the abstract data type stack. Currently we are using a linked list instead of the ubiquitous internal table.

Classes LCL_CALL_STACK, LCL_CALL_STACK_COMPACT


inherits from LCL_UML_STACK and implements the generation of messages, The very first call and the very last return are implemented as special cases.

Class LCL_SEQUENCE


method LIF_COLLECTOR~COLLECT( ) of class LCL_SEQUENCE converts the trace entries of type TS_SAT into message entries of type TS_MESSAGE referring to the actors dictionary MI_ACTORS. After the MT_TRACE list of message is filled, the message iterator is created and linked to an UML translator, we call the resulting object composition a Call Stack (class LCL_CALL_STACK or its loops aware subclass  LCL_CALL_STACK_COMPACT) that can use the TO_UML( ) method to traverse the call hierarchy and generate an LCL_DIAGRAM_TEXT class (UML sequence diagram in text mode).

The abstract LCL_UML class is used, concrete subclasses define the output format (default: subclass LCL_UML_PLANT for PlantUML output). Subclass LCL_MESSAGES_COMPACT creates an LCL_LOOP_COMPACTOR object in its constructor and uses it for loop compaction by calling the FOLD( ) method described below and by creating an object of class LCL_UML_CYCLES for the correct generation of UML combined fragments for loops.

Classes LCL_UML, LCL_UML_PLANT


Abstract class LCL_UML public methods generate valid UML sequence diagram statements in text form. The concrete subclass LCL_UML_PLANT is used for the default target (PlantUML). Other text formats are implemented (classes LCL_UML_GRAPH, LCL_UML_MSCEN) but not well tested. The result string with the complete UML code is passed back to class LCL_DIAGRAM_TEXT.

Classes LCL_DIAGRAM_TEXT, LCL_PLANT_UML_DIAGRAM


After generation, the abstract class LCL_DIAGRAM_TEXT orchestrates the output. Its subclass LCL_PLANT_UML_DIAGRAM either

  • saves the plain text source of the diagram using class LCL_FILE or

  • executes a local PlantUML installation to generate an image file or

  • encodes the diagram into an URL for the PlantUML server that will create an image.


The result is displayed in an ABAP browser window (standard class CL_ABAP_BROWSER).

Classes LCL_PATTERN, Interface LIF_CYCLES, Classes LCL_LOOP_COMPACTOR, LCL_TRACE_INDEX


Class LCL_PATTERN method DETECT_CYCLES( ) searches for verbatim repetition sequences in a trace. The cycle detector traverses the trace table and tries to locate the next occurrence of current entry in the rest of the trace. If a match is found, a second iterator starting at the position of this next occurrence is created for the same trace table. By comparing both iterators output we can find the longest matching sequence. This logic is used recursively to find loops inside loops. This simple matching approach cannot recognize cases with conditional flow in the loop.

The ABAP statement FIND FIRST OCCURENCE OF.. IN TABLE outperforms a combination of LOOP / READ TABLE statements. To benefit from this, the loop detection and compaction are performed on an index/dictionary table rather than on the trace data structure MT_TRACE. Class LCL_TRACE_INDEX implements the dictionary creation, the reverse conversion and the single loop detection step by delegating to the LCL_PATTERN class.

For each loop found, a loop entry of structure TS_CYCLE with



























Field
Description
START index of the starting entry of the repeated pattern (e.g., 1)
END index of the end entry of the repeated pattern (e.g., 4)
LAST index of the last entry of the repetition, when the loop is exited (e.g., 12)
TIMES number of loop counts (e.g., 3)


is passed to the COLLECT( ) method of interface LIF_CYCLES, allowing to limit the dependency between classes. Our implementation uses SKIP( ) instead of NEXT( ) to traverse entries and a CLONE( ) method to generate new iterators on the same table. Class LCL_LOOP_COMPACTOR implements the LIF_CYCLES interface and uses the loop entries to compact the trace table.

The compaction step in done both for in the dictionary table and in the loop entry table of class LCL_LOOP_COMPACTOR. Call LCL_MESSAGES_COMPACT initiates loop compaction by calling method LCL_LOOP_COMPACTOR->FOLD( ) that repeat loop detection and elimination until no further trace table size reduction is achieved.

Class LCL_PROGRESS_INDICATOR


calls to the ECHO( ) method are used for update information in the status line in processing intensive code.

Class LCL_CONFIGURATION


sets and retrieves user settings. Method QUERY( ) generates a popup customizing dialog and method GET( ) reads the current settings.

Test Suite


Automated Tests


The ABAP unit tests characterize the current system behavior and so provide a regression test suite to help maintain the code. About 20 ABAP unit test classes and 100 test methods currently achieve branch coverage of ca. 80%, so we are confident to have a high sensibility to change.

As all classes are local, all classes to be tested grant friendship to local interface LIF_UNIT_TEST. All ABAP unit test classes implement this interface to enable access to private sections of the code. Since no test double framework is available for local classes, some helper classes were created to enable behavior verification.

  • LCL_LOGGER

  • LCL_ABAP_TRACE_LOGGER

  • LCL_UML_LOGGER

  • LCL_FILE_NULL

  • LCL_SATRA_STATE


Test Case


The following report is from a SCN thread. Check the generated sequence diagram below
REPORT ZZZ_CLONE.

CLASS lcl_range DEFINITION.

  PUBLIC SECTION.

    TYPES tv_date TYPE d.

    DATA: mv_start TYPE tv_date,

          mv_stop TYPE tv_date.



    METHODS constructor

      IMPORTING iv_start TYPE tv_date

                iv_stop TYPE tv_date.

    METHODS create_copy

        IMPORTING iv_start TYPE tv_date OPTIONAL

                  iv_stop TYPE tv_date OPTIONAL

        RETURNING VALUE(ro_range) TYPE REF TO lcl_range.

ENDCLASS.



CLASS lcl_range IMPLEMENTATION.

  METHOD constructor.

    super->constructor( ).

    mv_start = iv_start.

    mv_stop = iv_stop.

  ENDMETHOD.



  METHOD create_copy.

    DATA lv_start TYPE tv_date.

    DATA lv_stop TYPE tv_date.



    WRITE:/ 'Called with START = ',  iv_start, ' STOP=',  iv_stop.

    IF iv_start IS NOT SUPPLIED.

      lv_start = mv_start.

    ELSE.

      lv_start = iv_start.

    ENDIF.

    IF iv_stop IS NOT SUPPLIED.

      lv_stop = mv_stop.

    ELSE.

      lv_stop = iv_stop.

    ENDIF.

    WRITE:/ 'Updated to START = ',  lv_start, ' STOP=',  lv_stop.

    CREATE OBJECT ro_range

      EXPORTING iv_start = lv_start

                iv_stop = lv_stop.

  ENDMETHOD.

ENDCLASS.



DATA go_2009_range TYPE REF TO lcl_range.

DATA go_copy TYPE REF TO lcl_range.



START-OF-SELECTION.

  CREATE OBJECT go_2009_range

    EXPORTING iv_start = '20090101'

              iv_stop = '20091221'.

* Clone

  go_copy = go_2009_range->create_copy( ).

* Extend range to end of 2010

  go_copy = go_2009_range->create_copy( iv_stop = '20101231' ).

* New start date: May 1st 2009

  go_copy = go_2009_range->create_copy( iv_start = '20090501' ).



 

The sequence diagram for a one line WRITE 'Hello!' program shows that much is going on under the hood:



This report is generated from report to display Short Dump Texts



Part of the UML diagram generation code as it is executed (Parse Trace Data)


Web Search for Existing Research Papers



Sharp and Rountev paper Interactive Exploration of UML Sequence Diagrams (2005) and their material propose the following filters for reverse-engineered sequence diagram I would like to implement:

  • Choose starting and ending message

  • Choose max. call stack depth

  • Focus on a single message

  • Filter out everything outside of a chosen fragment

6 Comments
Labels in this area
Popular Blog Posts