ABAP Trace to PlantUML Sequence Diagram
Some shortcomings of the SAP standard sequence diagram generation from ABAP run-time measurements are addressed:
- a secondary internal table index speeds up parsing
- a custom filter displays calls from standard to custom code
- loop compaction produces smaller diagrams
- the sequence diagram output in PlantUML Format is editable plain text .
Local installation of PlantUML is recommended but not mandatory as a web service is available. The JNet OCX is not needed.
- Code is available at github: Create UML Class and Sequence Diagrams from ABAP Code
- This content was first published 2014 as a SAP wiki. The link is not accessible and I believe a blog is a stable location for this document.
UML from ABAP Code
Transaction SAT profiles an ABAP program execution and provides analysis tools for performance, program flow and memory consumption. Measurements can be imported from other SAP systems using either download/upload or RFC. Since SAP Netweaver 7.02, an UML sequence diagram can be generated if the measurement was recorded without aggregation.
- Start transaction SAT -> Evaluate -> Double Click on a measurement; on the next screen press F8 (Display measurement as UML sequence diagram).
A sequence diagram will be displayed if the JNet OCX component is installed (e.g. with SAP GUI). This feature was never fully delivered according to SAP OSS Note 1744063: development for this function has not yet finished and therefore contains some serious errors including program terminations.
Design-Time Sequence Diagrams
At design time, sequence diagrams are usually created manually before the code is implemented to visualize collaboration aspects of the desired system actors. Design time sequence diagrams accelerate the discussion on stable interfaces (e.g. compared with this SCN thread OO Analysis) before the system is built. Sandi Metz says that each message requires that the sender know things about the receiver. This knowledge creates dependencies between the two and these dependencies stand in the way of change
Let us compare two diagrams generated for reports from the thread Classical way to ABAP OO style of coding with the same functionality.
- The first diagram reveals the logic: GET_DATA( ) selects some business data, and then DISPLAY( ) implements the output / user interface.
- The second program attempts to separate concerns: the new actor LCL_MAIN exposes the business logic as a service requested by sending a START( ) message. The design pattern here is to evaluate which parts of the system are likely to change and to isolate those parts in new actors with carefully crafted interfaces.
With a stable interface, caller and called actors can change independently. As long as they abide by the contract, e.g. method START( ), there is no ripple effect of change through the system.
It is tempting to create layers of abstractions everywhere to increase flexibility, I made a more complex proposal in the use case above. The designer must ultimately decide if the increased complexity (new classes) is justified.
Sequence Diagrams from Run-time Data
Our aim is to automate the sequence diagrams generation from run-time measurements. When the creation cost is low, the feedback loop is short, and we can routinely use sequence diagrams to appreciate the impact of code changes on the overall system design.
Object oriented ABAP code defines classes while the application behavior is better explained by messages exchanged between run-time objects. The appealing visualization revealing actual communication patterns makes it easier to discuss separation of concern, stable interfaces and simpler protocols for objects exposing too much of their inner structure.
Enhance Sequence Diagrams Output
SAP has provided an easy enhancement path for the UML class diagram output (subclass class CL_UML_CLASS_DECORATOR). Some hard coded limitations in the ATRA_UML_DECIDER function module and the CL_ATRA_FACTORY class must be overridden to enhance sequence diagrams by subclassing CL_ATRA_UML (implementation of the IF_ATRA_UML_TOOL interface). I have been using Chris Paine’s code and I liked it very much. I now believe his breakthrough design decision was to delegate the graphical representation to an external tool. His code saves a sequence diagram in the format of the free PlantUML tool that requires Java on your machine. PlantUML syntax is human readable and editable plain text. A Chrome browser extension displays the output as soon as the text file is updated.
My first real contribution was to implement loop compaction, i.e., use UML combined fragments to express loops. This is done recursively until no further gain is achieved; it is gratifying to generate a small diagram with LOOP statements from large traces.
A future improvement could be to build a dictionary of non-adjacent blocks and increase trace compaction by using conditionals ALT or OPT combined fragments. I also found the trace parsing could be accelerated by using a secondary internal table index, so the current code replaces the standard sequence diagram display logic completely instead of extending it. I could consume the PlantUML web service in ABAP.
This leads to a short feedback loop and learning is enjoyable… if the image is not too large.
Avoiding Large Traces
Transaction SAT can generate very large traces, with much more than 1 million entries, that are impossible to process: the evaluation is extremely slow or fails with a timeout. This, I presume, is why the size limit is so prominently featured in the variant customizing. But it does not take the largest trace to generate diagrams too large to deliver useful insights into the design. Care is needed if we want to selectively trace program parts and so restrict the generated diagram to parts we are interested in. We can start a trace
- In Dialog – the measurement variant can be setup to switch measurement off and on manually or with the statement SET RUN TIME ANALYZER ON/OFF, to discard some processing blocks and limit the measurement to given program components.
- In a parallel session – the measurement is then always started/stopped manually
- In a background session (for another user/service) – cf. Chris Paine’s blog.
- While evaluating an existing trace, we can use the filter dialog (Ctrl+F1) to narrow the processing to selected actors or messages. Chris Paine proposed a custom code filter that also displays calls from standard to custom code.
- We can optimize the diagram generation for speed or size: Following advice from a PlantUML developer, I introduced a scale factor. Scale value 0.5 reduces the generated image size by 1/4.
Class CL_ATRA_UML_HANDLING controls the generation of an UML sequence diagram from a measurement without aggregation in transaction SAT. Method SHOW_SEQ_DIAGRAM will call method DISPLAY_UML( ) of local class LCL_UML_HANDLING. Three steps are needed to override the standard behavior:
- Create a new Include YY_SATRA_INCLUDE using the code available from github.
- In class builder for class CL_ATRA_UML_HANDLING Goto -> Local Definitions/Implementations -> Local Definitions/Implementations Ctrl+Shift+F6. The definition/implementation of local class LCL_UML_HANDLING is displayed. At the bottom of the source code, implement an implicit enhancement and insert the logic from the include:
ENHANCEMENT 2 YY_SATRA_SEQUENCE. "active version INCLUDE YY_SATRA_INCLUDE. ENDENHANCEMENT.
- The last step is to override the call in method SHOW_SEQ_DIAGRAM( ) using an enhancement:
ENHANCEMENT 1 YY_SATRA_SEQUENCE. "active version DATA lx_error TYPE REF TO cx_dynamic_check. TRY. lcl_sequence=>to_diagram( lcl_configuration=>query( ) )->output( ). CATCH cx_dynamic_check INTO lx_error. MESSAGE lx_error TYPE 'I' DISPLAY LIKE 'E'. "#EC CI_USE_WANTED ENDTRY. RETURN. ENDENHANCEMENT.
- After activation, your system behavior will have changed. A new customizing popup appears while displaying a sequence diagram.
Trace Analysis Steps
- Query ABAP execution trace created by transaction SAT without aggregation
- Parse call hierarchy in trace, apply custom filters
- Convert filtered trace to sequence diagram in text form
- Save UML as text and generate and/or display image.
Classes LCL_ABAP_TRACE and LCL_TRACE, Interface LIF_COLLECTOR
Class LCL_ABAP_TRACE parses the call hierarchy (form routines, methods, function modules, screen flow…) and actors (programs, function groups, classes, logical databases…) of an ABAP trace stored in static attributes of standard class CL_ATRA_TOOL_SE30_MAIN, i.e. the internal tables
- IT_TRACEPROG – Table of Recorded Program Names
- IT_TRACETEXT – Table of TRACE Text Elements
- IT_TRACEMETH – Table of Recorded Method Names
- IT_AUSTAB_HIER – Call Hierarchy table with All TRACE Information
|ID||Indicator describes the type of the recorded event (call method, perform, call function, call screen, database operation, message…)|
|FROM_LEVEL||Call level in the call hierarchy (an integer)|
|CALLER||caller object – Source of the message (program, function group, class, etc.)|
|CALLED||called object – Receiver (target) of the message|
|AUS_TABIX||Index for table access|
|SYSTEM||System flag ” only used in custom filter|
This structure is passed to any collector class like LCL_SEQUENCE implementing the LIF_COLLECTOR interface. A collector is responsible for applying custom filter. The COLLECT( ) method must returns ABAP_FALSE if the entry was rejected (filtered out) so the LCL_TRACE method can adjust the whole call hierarchy. Some helper classes were introduced:
- LCL_BAG – Abstract data type with the interface ADD/REMOVE/CONTAINS simplifies the filter logic
- LCL_CLASS_NAME to transform between technical and external ABAP class name
Classes LCL_ACTORS, Interface LIF_ACTORS
The rest of the code uses the term Actor according to the sequence diagram definition to denote any object that sends or receives messages. The collection of actors (or actor dictionary) is accessed via a LIF_ACTORS interface that defines:
- method NEW_PATH( ) saves the caller and called object from structure TS_SAT in the actor collection. New entries are created and existing entries are retrieved from the collection of actors. A unique key is returned for each actor, so a pair of actor indices (a PATH) is returned by the NEW_PATH( ) method for the caller and called objects.
- the short text of a message (mostly determined by the called object/actor) is formatted using the SHORT_TEXT( ) method
- method LIFELINES( ) returns the complete list (internal table) of sequence diagram lifelines. It is needed at initialization and completion of the sequence diagram.
Class LCL_ACTORS implements the LIF_ACTORS interface. The actors are stored in a sorted internal table MT_ACTOR with the unique secondary key OBJ_NR for performance. While comparing actors, a special treatment is needed for entries of type object (the object oriented instance of a class), since each object instance has its separate identity – cf method INDEX_OF( ).
Interface LIF_TRACE_FILTER, Classes LCL_FILTER, LCL_FILTER_NULL, LCL_FILTER_CUSTOM
The filter logic is abstracted to interface LIF_TRACE_FILTER to make it easy to try new ideas. The easiest way to create a new custom filter is to subclass class LCL_FILTER and redefine the method ACCEPTS( ) with a custom logic. Adjust the factory method NEW( ) of class LCL_FILTER to inject the new logic and no additional changes to the collector in LCL_SEQUENCE would be needed. We provide three filter implementations:
- Class LCL_FILTER_NULL is a dummy filter without impact. It is only used if the Loops compacted checkbox is disabled and if no entries exist in the Customer namespace field
- Class LCL_FILTER rejects some low-level object identifiers. It is used if the Loops compacted checkbox is disabled and the Customer namespace field is maintained.
- The default implementation in class LCL_FILTER_CUSTOM applies a Customer namespace filter but also keeps actors from other name spaces calling that trigger messages to custom code (Chris Paine’s logic).
Note: A lifeline without short text will be displayed if the actor is filtered out but the message is not.
is an iterator pattern implementation that traverses the collected messages for UML sequence diagram generation. The canonical methods are
- HAS_NEXT( ) – check for availability of an entry and
- NEXT( ) – get the next entry for an internal table.
We further expose methods
- SKIP( ) – skip a number of entries (default 1)
- IS_FIRST( ),
- FIRST_LEVEL( ) and NEXT_LEVEL( ) to return the current call depth.
Classes LCL_STACK, LCL_UML_STACK
implements the stack (last in-first out queue) methods PUSH( ) and POP( ) to store call levels (structure TS_CALL_LEVEL with fields ACTOR_KEY and FROM_LEVEL).This implementation always updates flag MV_EMPTY but it does never throw any exception; instead, it returns an initial value when the stack is empty.
The subclass LCL_UML_STACK uses the stack to implement a call stack with methods CALL( ) and the RETURN( ) behavior depending on the call level. Note the LCL_STACK was only extracted to try a different implementation of the abstract data type stack. Currently we are using a linked list instead of the ubiquitous internal table.
Classes LCL_CALL_STACK, LCL_CALL_STACK_COMPACT
inherits from LCL_UML_STACK and implements the generation of messages, The very first call and the very last return are implemented as special cases.
method LIF_COLLECTOR~COLLECT( ) of class LCL_SEQUENCE converts the trace entries of type TS_SAT into message entries of type TS_MESSAGE referring to the actors dictionary MI_ACTORS. After the MT_TRACE list of message is filled, the message iterator is created and linked to an UML translator, we call the resulting object composition a Call Stack (class LCL_CALL_STACK or its loops aware subclass LCL_CALL_STACK_COMPACT) that can use the TO_UML( ) method to traverse the call hierarchy and generate an LCL_DIAGRAM_TEXT class (UML sequence diagram in text mode).
The abstract LCL_UML class is used, concrete subclasses define the output format (default: subclass LCL_UML_PLANT for PlantUML output). Subclass LCL_MESSAGES_COMPACT creates an LCL_LOOP_COMPACTOR object in its constructor and uses it for loop compaction by calling the FOLD( ) method described below and by creating an object of class LCL_UML_CYCLES for the correct generation of UML combined fragments for loops.
Classes LCL_UML, LCL_UML_PLANT
Classes LCL_DIAGRAM_TEXT, LCL_PLANT_UML_DIAGRAM
After generation, the abstract class LCL_DIAGRAM_TEXT orchestrates the output. Its subclass LCL_PLANT_UML_DIAGRAM either
- saves the plain text source of the diagram using class LCL_FILE or
- executes a local PlantUML installation to generate an image file or
- encodes the diagram into an URL for the PlantUML server that will create an image.
The result is displayed in an ABAP browser window (standard class CL_ABAP_BROWSER).
Classes LCL_PATTERN, Interface LIF_CYCLES, Classes LCL_LOOP_COMPACTOR, LCL_TRACE_INDEX
Class LCL_PATTERN method DETECT_CYCLES( ) searches for verbatim repetition sequences in a trace. The cycle detector traverses the trace table and tries to locate the next occurrence of current entry in the rest of the trace. If a match is found, a second iterator starting at the position of this next occurrence is created for the same trace table. By comparing both iterators output we can find the longest matching sequence. This logic is used recursively to find loops inside loops. This simple matching approach cannot recognize cases with conditional flow in the loop.
The ABAP statement FIND FIRST OCCURENCE OF.. IN TABLE outperforms a combination of LOOP / READ TABLE statements. To benefit from this, the loop detection and compaction are performed on an index/dictionary table rather than on the trace data structure MT_TRACE. Class LCL_TRACE_INDEX implements the dictionary creation, the reverse conversion and the single loop detection step by delegating to the LCL_PATTERN class.
For each loop found, a loop entry of structure TS_CYCLE with
|START||index of the starting entry of the repeated pattern (e.g., 1)|
|END||index of the end entry of the repeated pattern (e.g., 4)|
|LAST||index of the last entry of the repetition, when the loop is exited (e.g., 12)|
|TIMES||number of loop counts (e.g., 3)|
is passed to the COLLECT( ) method of interface LIF_CYCLES, allowing to limit the dependency between classes. Our implementation uses SKIP( ) instead of NEXT( ) to traverse entries and a CLONE( ) method to generate new iterators on the same table. Class LCL_LOOP_COMPACTOR implements the LIF_CYCLES interface and uses the loop entries to compact the trace table.
The compaction step in done both for in the dictionary table and in the loop entry table of class LCL_LOOP_COMPACTOR. Call LCL_MESSAGES_COMPACT initiates loop compaction by calling method LCL_LOOP_COMPACTOR->FOLD( ) that repeat loop detection and elimination until no further trace table size reduction is achieved.
calls to the ECHO( ) method are used for update information in the status line in processing intensive code.
sets and retrieves user settings. Method QUERY( ) generates a popup customizing dialog and method GET( ) reads the current settings.
The ABAP unit tests characterize the current system behavior and so provide a regression test suite to help maintain the code. About 20 ABAP unit test classes and 100 test methods currently achieve branch coverage of ca. 80%, so we are confident to have a high sensibility to change.
As all classes are local, all classes to be tested grant friendship to local interface LIF_UNIT_TEST. All ABAP unit test classes implement this interface to enable access to private sections of the code. Since no test double framework is available for local classes, some helper classes were created to enable behavior verification.
The following report is from a SCN thread. Check the generated sequence diagram below
REPORT ZZZ_CLONE. CLASS lcl_range DEFINITION. PUBLIC SECTION. TYPES tv_date TYPE d. DATA: mv_start TYPE tv_date, mv_stop TYPE tv_date. METHODS constructor IMPORTING iv_start TYPE tv_date iv_stop TYPE tv_date. METHODS create_copy IMPORTING iv_start TYPE tv_date OPTIONAL iv_stop TYPE tv_date OPTIONAL RETURNING VALUE(ro_range) TYPE REF TO lcl_range. ENDCLASS. CLASS lcl_range IMPLEMENTATION. METHOD constructor. super->constructor( ). mv_start = iv_start. mv_stop = iv_stop. ENDMETHOD. METHOD create_copy. DATA lv_start TYPE tv_date. DATA lv_stop TYPE tv_date. WRITE:/ 'Called with START = ', iv_start, ' STOP=', iv_stop. IF iv_start IS NOT SUPPLIED. lv_start = mv_start. ELSE. lv_start = iv_start. ENDIF. IF iv_stop IS NOT SUPPLIED. lv_stop = mv_stop. ELSE. lv_stop = iv_stop. ENDIF. WRITE:/ 'Updated to START = ', lv_start, ' STOP=', lv_stop. CREATE OBJECT ro_range EXPORTING iv_start = lv_start iv_stop = lv_stop. ENDMETHOD. ENDCLASS. DATA go_2009_range TYPE REF TO lcl_range. DATA go_copy TYPE REF TO lcl_range. START-OF-SELECTION. CREATE OBJECT go_2009_range EXPORTING iv_start = '20090101' iv_stop = '20091221'. * Clone go_copy = go_2009_range->create_copy( ). * Extend range to end of 2010 go_copy = go_2009_range->create_copy( iv_stop = '20101231' ). * New start date: May 1st 2009 go_copy = go_2009_range->create_copy( iv_start = '20090501' ).
The sequence diagram for a one line WRITE ‘Hello!’ program shows that much is going on under the hood:
This report is generated from report to display Short Dump Texts
Part of the UML diagram generation code as it is executed (Parse Trace Data)
Web Search for Existing Research Papers
- Extracting Sequence Diagram from Execution Trace of Java Program
- Improving The Scalability of Tools Incorporating Sequence Diagram Visualizations of Large Execution Traces
- Visualisations of Execution Traces
- Sequence Diagram Slicing
- Execution Trace Analysis through Massive Sequence and Circular Bundle Views or http://www.st.ewi.tudelft.nl/~cornel/pub/JSS08.pdf
- Static Control-Flow Analysis for Reverse Engineering of UML Sequence Diagrams
- A Fully Dynamic Approach to the Reverse Engineering of UML Sequence Diagrams
- Execution Trace Abstraction based on Meta Patterns Usage
- Extracting Sequence Diagrams from Execution Traces Using Interactive Visualization
- A Metamodel for Dynamic Information Generated from Object-Oriented Systems or http://dmct.dorsal.polymtl.ca/sites/dmct.dorsal.polymtl.ca/files/Sept17-SOTA-TraceAbstractionTechniques.pdf
- Techniques to Simplify the Analysis of Execution Traces for Program Comprehension
- ANALYTIC VARIATIONS ON THE COMMON SUBEXPRESSION PROBLEM
Sharp and Rountev paper Interactive Exploration of UML Sequence Diagrams (2005) and their material propose the following filters for reverse-engineered sequence diagram I would like to implement:
- Choose starting and ending message
- Choose max. call stack depth
- Focus on a single message
- Filter out everything outside of a chosen fragment
Good work and thanks for sharing!
kudos for this well crafted blog.
You stated that
How to do that? I seems that linked git repo isn't an abapGit repository.
you are right, it is not possible yet. (cf. to Re-Integrating the Forks? · Issue #3 · nomssi/ABAP-to-PlantUML (github.com). I will try to create a repository with the latest code and update the blog.
I apologize for the misleading text.
Hi Jacques, is there any way to extend the size limit? If you try to produce a large sequence diagram you either have to scale it down or it is cut off. Do you have any idea? This seems to be a known issue, but there might be some ways to set environmental parameters to override the PLANTUML standard settings. Is there any way to forward these parameters from the ABAP source?
PS Good stuff!
I just checked the PlantUML site. There is currently a problem when the HTTP request is too large, so they implemented a trick to send a compressed request.
Help: Compressing long markup with a low-code tool - PlantUML Q&A
I will check this feature and report here later. I will also implement the parameters "V-pages and H-pages".
For now, your best bet is to setup a local PlantUML server on your machine. I assume you will have a slightly larger cut-off limit then the one on the online webserver.
I checked and the HTTP request generated is already compressed, so the trick I was referring to is not helpful in this context. For the purpose of generating larger images, the only successully workaround I found was to use a local copy of the PlantUML jar file.
I have just updated the YY_SATRA_INCLUDE include to enable the parameters H-pages and V-pages in sequence diagrams, making it possible to generate HxV tiles instead of a single large image.
ABAP-to-PlantUML/yy_satra_include.abap at master · nomssi/ABAP-to-PlantUML (github.com)
Jacques Nomssi Nzali