The SQL Trace (ST05) Quick and Easy
The SQL Trace, which is part of the Performance Trace (transaction ST05), is the most important tool to test the performance of the database. Unfortunately, information on how to use the SQL Trace and especially how to interpret its results is not part of the standard ABAP courses. This weblog tries to give you a quick introduction to the SQL Trace. It shows you how to execute a trace, which is very straightforward. And it tells how you can get a very condensed overview of the results–the SQL statements summary–a feature that many are not so familiar with. The usefulness of this list becomes obvious when the results are interpreted. A short discussion of the ‘database explain’ concludes this introduction to the SQL Trace.
1. Using the SQL Trace
Using the SQL trace is very straightforward:
- Call the SQL trace in a second mode
- Make sure that your test program was executed at least once, or even better, a few times, to fill the buffers and caches. Only a repeated execution provides reproducible trace results. Initial costs are neglected in our examination
- Start the trace
- Execute your test program in the first mode
- Switch off the trace. Note, that only one SQL trace can be active on an application server, so always switch your trace off immediately after your are finished.
- Display the trace results
- Interpretation of the results
Note, the trace can also be switched on for a different user.
=> In this section we showed how the SQL trace is executed. The execution is very straightforward and can be performed without any prior knowledge. The interpretation of the results, however, requires some experience. More on the interpretation will come in the next section.
2. Trace Results – The Extended Trace List
When the trace result is displayed the extended trace list comes up. This list shows all executed statements in the order of execution (as extended list it includes also the time stamp). One execution of a statement can result in several lines, one REOPEN and one or several FETCHES. Note that you also have PREPARE and OPEN lines, but you should not see them, because you only need to analyze traces of repeated executions. So, if you see a PREPARE line, then it is better to repeat the measurement, because an initial execution has also other effects, which make an analysis difficult.
If you want to take the quick and easy approach, the extended trace list is much too detailed. To get a good overview you want to see all executions of the same statement aggregated into one line. Such a list is available, and can be called by the menu ‘Trace List -> Summary by SQL Statements’.
=> The extended trace list is the default result of the SQL Trace. It shows a lot of and very detailed information. For an overview it is much more convenient to view an aggregated list of the trace results. This is the Summarized SQL Statements explained in the next section.
3. Trace Results – Summarized SQL Statements
This list contains all the information we need for most performance tuning tasks.
The keys of the list are ‘Obj Name’ (col. 12), i.e. table name, and ‘SQL Statement’ (col. 13). When using the summarized list, keep the following points in mind:
- Several coding positions can relate to the same statement:
- The statement shown can differ from its Open SQL formulation in ABAP.
- The displayed length of the field ‘Statement’ is restricted, but sometimes the displayed text is identical.
- In this case, the statements differ in part that is not displayed.
The important measured values are ‘Executions’ (col. 1), ‘Duration’ (col. 3) and ‘Records’ (col. 4). They tell you how often a statement was executed, how much time it needed in total and how many records were selected or changed. For these three columns also the totals are interesting; they are displayed in the last line. The other totals are actually averages, which make them not that interesting.
Three columns are direct problem indicators. These are ‘Identical’ (col. 2), ‘BfTp’ (col. 10), i.e. buffer type, and ‘MinTime/R.’ (col. 8), the minimal time record.
Additional, but less important information is given in the columns, ‘Time/exec’ (col. 5), ‘Rec/exec’ (col. 6), ‘AvgTime/R.’ (col. 7), ‘Length’ (col. 9) and ‘TabType’ (col. 11).
For each line four functions are possible:
- The magnifying glass shows the statement details; these are the actual values that were used in the execution. In the summary the values of the last execution are displayed as an example.
- The ‘DDIC information’ provides some useful information about the table and has links to further table details and technical settings.
- The ‘Explain’ shows how the statement was processed by the database, particularly which index was used. More information about ‘Explain’ can be found in the last section.
- The link to the source code shows where the statement comes from and how it looks in OPEN SQL.
=> The Statement summary, which was introduced here, will turn out to be a powerful tool for the performance analysis. It contains all information we need in a very condensed form. The next section explains what checks should be done.
4. Checks on the SQL Statements
For each line the following 5 columns should be checked, as tuning potential can be deduced from the information they contain. Select statements and changing database statements, i.e. inserts, deletes and updates, can behave differently, therefore also the conclusions are different.
For select statements please check the following:
- Entry in ‘BfTy’ = Why is the buffer not used?
The tables which are buffered, i.e. with entries ‘ful’’ for fully buffered, ‘gen’ for buffered by generic region and ‘sgl’ for single record buffer, should not appear in the SQL Trace, because they should use the table buffer. Therefore, you must check why the buffer was not used. Reasons are that the statement bypasses the buffer or that the table was in the buffer during the execution of the program. For the tables that are not buffered, but could be buffered, i.e. with entries starting with ‘de’ for deactivated (‘deful’, ‘degen’, ‘desgl’ or ;deact’) or the entry ‘cust’ for customizing table, check whether the buffering could not be switched on. - Entry in ‘Identical’ = Superfluous identical executions
The column shows the identical overhead as a percentage. Identical means that not only the statement, but also the values are identical. Overhead expresses that from 2 identical executions one is necessary, and the other is superfluous and could be saved. - Entry in ‘MinTime/R’ larger than 10.000 = Slow processing of statement
An index-supported read from the database should need around 1.000 micro-seconds or even less per record. A value of 10.000 micro-seconds or even more is a good indication that there is problem with the execution of that statement. Such statements should be analyzed in detail using the database explain, which is explained in the last section. - Entry in ‘Records’ equal zero = No record found
Although this problem is usually completely ignored, ‘no record found’ should be examined. First, check whether the table should actually contain the record and whether the customizing and set-up of the system is not correct. Sometimes ‘No record found’ is expected and used to determine program logic or to check whether keys are still available, etc. In these cases only a few calls should be necessary, and identical executions should absolutely not appear. - High entries in ‘Executions’ or ‘Records’ = Really necessary?
High numbers should be checked. Especially in the case of records, a high number here can mean that too many records are read.
For changing statements, errors are fortunately much rarer. However, if they occur then they are often more serious:
- Entry in ‘BfTy’ = Why is a buffered table changed?
If a changing statement is executed on a buffered statement, then it is questionable whether this table is really suitable for buffering. In the case of buffered tables, i.e entries ‘ful’, ‘gen’ or ’sgl’’, it might be better to switch off the buffering. In the case of bufferable tables, the deactivation seems to be correct. - Entry in ‘Identical’ = Identical changes must be avoided
Identical executions of changing statements should definitely be avoided. - Entry in ‘MinTime/R’ larger than 20.000 = Changes can take longer
Same argument as above just the limit is higher for changing statements. - Entry in ‘Records’ equal zero = A change with no effect
Changes should also have an effect on the database, so this is usually a real error which should be checked. However, the ABAP modify statement is realized on the database as an update followed by an insert if the record was not found. In this case one statement out of the group should have an effect. - High entries in ‘Executions’ and ‘Records’ = Really necessary?
Same problems as discussed above, but in this case even more serious.
=> In this section we explained detailed checks on the statements of the SQL Statement Summary. The checks are slightly different for selecting and changing statements. They address questions such as why a statement does not use the table buffer, why statements are executed identically, whether the processing is slow, why a statement was executed but no record was selected or changed, and whether a statement is executed too often or selects too many records.
5. Understanding the Database Explain
The ‘database explain’ should show the SQL statement as it goes to the database, and the execution plan on the database. This view has a different layout for the different database platforms supported by SAP, and it can become quite complicated if the statement is complicated.
In this section we show as an example the ‘Explain’ for a rather simple index-supported table access, which is one of the most common table accesses:
- The database starts with step 1, index unique scan DD02L~0, where the three fields of the where-condition are used to find a record on the index DD02L~0 (‘~0’ denotes always the primary key).
- In step 2, table access by index rowed DD02L, the rowid is taken from the index to access the record in the table directly.
Some databases display the execution plan in a graphical layout, where a double-click on the table gives additional information, as shown on the right side. There the date of the last statistics update and the number of records in the table are displayed. Also all indexes are listed with their fields and the number of distinct values for each field, with this information it is possible to calculate the selectivity of an index.
From this example you should understand the principle of the ‘Explain’, so that you can also understand more complicated execution plans. Some database platforms do not use graphical layouts and are a bit harder to read, but still show all the relevant information.
=> In this last section we showed an example of a database explain, which is the only way to find out whether a statement uses an index, and if so, which index. Especially in the case of a join, it is the proper index support that determines whether a statement needs fractions of seconds or even minutes to be finished.
Please try the SQL Trace by yourself. If there are any questions or problems, your feedback is always welcome!
Further Reading: Performance-Optimierung von ABAP-Programmen (in German!)
More information on performance topics can be found in my new textbook on performance (published Nov 2009). However please note, that it is right now only available in German.
Chapter Overview:
- Introduction
- Performance Tools
- Database Know-How
- Optimal Database Programming
- Buffers
- ABAP – Internal Tables
- Analysis and Optimization
- Programs and Processes
- Further Topics
- Appendix
In the book you will find detailed descriptions of all relevant performance tools. An introduction to database processing, indexes, optimizers etc. is also given. Many database statements are discussed and different alternatives are compared. The resulting recommendations are supported by ABAP test programs which you can download from the publishers webpage (see below). The importance of the buffers in the SAP system are discussed in chaptr five. Of all ABAP statements mainly the usage of internal tables is important for a good performance. With all the presented knowledge you will able to analyse your programs and optimize them. The performance implications of further topics, such as modularisation, workprocesses, remote function calls (RFCs), locks & enqueues, update tasks and prallelization are explained in the eight chapter.
Even more information – including the test programs – can be found on the webpage of the publisher.
I would recommend you especially the examples for the different database statements. The file with the test program (K4a) and necessary overview with the input numbers (K4b) can even be used, if you do not speak German!
Greetings,
Blag.
Now why can't SAP provide this kind of context for it customers instead of making them figure out how the tools work?
I agree it is not easy to gain knowledge on Performance tools such as SQL Trace, but usage of ST05 is covered in both the standard ABAP Academy program(TAW10 part 2) as well as standard course BC490 - Performance Programming in ABAP.
Keep writing 🙂
Regards,
Dushyant Shetty
Your blog is very good and easy to understand.
Boes,I have two questions to ask you,they are:
1.How find the creation date of any database table?As it creation date is not maintained in attributes of table too,we can only see the "last changed date".I tried this by SQL trace,but no use.....please help me.
2.In SQL Trace, in "duration column" genrally all the rows are marked as YELLOW.But I have seen few rows(2 to 3) are marked as PINK.May I know the significance of those values ?
Waiting for your reply....
Please send the reply to "raghavender.vadakattu@yash.com"
Thanks & Regards,
V.Raghavender.
I'm executing SELECT on table BFOD_A and BFOD_AB.
The SELECT has a PACKAGE SIZE of 200 but the SQL recs = 275. I was expecting each retrieval of 200 records.
Is the OPEN/RE-OPEN the actual time taken by the database to execute the SQL and the FETCH the transfer of these records from the DB to the application server?
I'm trying to understand why my program is taking too long to run and need an understanding if the high cost is the DB or transfer from the DB to the app server.
Also, on another table I've seen an OPEN (count 1000 recs) on a table and inmmediately followed by numerous consecutive FETCH's (Count = 140 * 7 plus 20) on the same table.
What is DB activity and watch is the app. server activity?
Thanks.
Soyab
The OPEN/REOPEN is very fast. That is only the connection to the database.
The FETCH is the search and transfer of the data.
The column REC shows how many records are transfered per fetch.
Why a PACKAGE SIZE of 200? Package sizes are usually much higher, about 10.000. And package size determines what you want to process in ABAP in your first step. The number of records per FETCH is not influenced by package size, it is limited by the block size of 32k divided by field length of a record, i.e. the number of records which fit into 32k.
> I'm trying to understand why my program is
> taking too long to run.
Look at the EXPLAIN !!!!! Time is determined by the index, all other effects are usually less important.
> an OPEN (count 1000 recs) on a table
there is no number in the OPEN line, what do you mean?
Siegfried
SAP provides some classes as mentioned and also a documentation. However, I thought it is better to provide a shorter introduction where really only the most important topics are discussed.
@Raghavender
1. Creation date of a table, what should this information be good for? I don't know whether it is kept somewhere, I have my doubts.
2. Pink colored duration times, only in extended list, these are just times above a limit. Not very helpful. You have to see how many records a selected in total by a statement and problems are indicated by the time needed to select one record, because a select fetching 1000 records can of course take longer than one getting only 1.
Siegfried
You have really demystified the topic
Congrats
Many Thanks
Aby
Seasons Greetings from Singapore
Please consider new option in filter "log statements with duration great than seconds" xyz
Thus even productive systems can be traced after go-live or support package upgrade or hardware/db upgrade for a longer time with minimum impact
Best Regards, Kishore.
Regards,
Sean.
no, the ST05 will only trace on same application server. It will capture RFCs if they run under the same user ID and during the period when the trace is active.
There is another tool, the ST30 (T not E) which does what you want. It is mainly used with ECATTs which run the process. Then it automatically activates the ST05 in all remote systems and collects the trace summaries into the guidance system. Please search for 'ST30 - Global Performance Analysis' for details.
Siegfried
would you know how best to work out how many ST05 trace records can be stored, say per MB of trace file. Our systems have the default values of 10 files at 16MB each but I've been asked how many trace file records this would allow before overwrite occurs.
Regards,
Brian.
I do not know these details, but I found someone, who should know.
The average size of a trace records is 1200 byte.
Of there is some variance depending on the statement.
kind regards Siegfried
Nice blog.
We can know the overview of ST05.
Thanks & Regards,
Supriya.
Thank you very much Siegfried Boes.
This was so nice going through.
Best wishes..
Nice Blog
Very concise and easy to understand
Hi,
That is a very nice blog, ST05 features are explained in a simple but solid way, thank you
On of the best descriptions I could find.
I just miss information about how to explain Execution plan figures, like in example below:
INDEX RANGE SCAN GLPCA~3
( Estim. Costs = 1 , Estim. #Rows = 1 )
Search Columns: 3
Estim. CPU-Costs = 5.114 Estim. IO-Costs = 1
Access Predicates Filter Predicates
Could you please explain what are underlined terms?
Hallo Adam,
what you ask is far beyond the scope of this short blog and it is actually highly overestimated.
If you are able to read German then you will find many examples for explains in the my book (reference above):
Your example roughly means that the optimizer expects 1 row to be returned by the access and that index 3 for table GLPCA is used. The CPU costs and IO costs are only interesting in comparisions.
kind regards Siegfried
Hello Boes,
Superb blog....... Salute you for knowledge sharing....................
This blog provides me lot of Performance tuning information
Thanks
Sumit
Please accept my apology for using obsolute word.......I
Nice information, Thanks.
A nice informative blog.
Dos any one have new documentation for the same? We do not have such screen and GUI in EHP5 .........................So this documentation is obsolete in higher version..
Please do not confuse blogs and documentation.
There is of course the documentation available for the latest releases, which you can find in the usual resources.
The content of this blog is still valid, if you adapt the layout to your more recent release.
So 'obsolete' is a it harsh.
best regards Siegfried
Hi Siegfried,
Thanks for the excellent blog on ST05. The analysis of the trace results is not explained anywhere and I used to struggle to come to a conclusion after doing a trace. Thanks once again for explaining in detail the analysis of the trace.
Thanks & Regards,
Kunal.
very well written... a lot of "good to know" stuff in there...
Hi Siegfried,
In the section Using the SQL trace, what do you mean by second mode in step 1 and first mode in step 4.
1.Call the SQL trace in a second mode
4. Execute your test program in the first mode
Hi Midhun,
Second mode is nothing but new session ( second session).
Regards,
Vamshi.
Dear Siegfried Boes,
Thanks a lot for such a lucid explanation of ST05, Even Functional consultants can follow your explanation. Much appreciated for your help to the community.
Thanks
Regards
Pankaj Agrawal