Analyzing E2E performance using Introscope Transaction Tracer, SAP Passport, and the BI Platform Support Tool
Sometimes understanding exactly what part of a particular workflow is contributing to slow response times can be challenging. In the BI Platform landscape, there are many components and variables which contribute to the response times that end users are experiencing. Thanks to the tight integration of SAP BI Platform 4.x and CA APM Introscope, we have a very useful and powerful tool that we can use to see under the hood and provide response times for very granular aspects of the reporting workflow. This tool is the Transaction Tracer and is included in the set of monitoring and diagnostic tools that Introscope provides. The Transaction Tracer is a feature of the Introscope Workstation that allows you to record transactions across the landscape and then filter these transactions based on SAP Passport which is generated by the SAP Client Plug-in. After the trace is generated and filtered, the data can be quickly analyzed in great detail using the Workstation’s rich graphical user interface.
In this article, I will demonstrate how to generate and analyze an end to end trace using the Introscope Transaction Tracer and SAP Passport without the use of SAP Solution Manager. In place of SAP Solution Manager, the SAP BI Platform Support Tool (a free utility for BI administrators) will provide the capabilities needed to generate the query which will be submitted to Introscope. If you are interested in this topic but are instead using SAP Solution Manager (rather than CA APM Introscope in stand-alone mode), check out this wiki page for a tutorial. If you do not have Introscope configured for BI Platform 4.x yet and are looking for instructions and a tutorial, refer to the following article A complete guide to setup CA APM Introscope 9 for monitoring SAP BI Platform 4.1 on Windows [PART 1/2]
Validate that metrics are being collected
The first step that should be done prior to generating the trace is to confirm that all the necessary metrics are being collected for the workflow you are tracing. In this example, we will be tracing the time that it takes to refresh a Web Intelligence document against the E-fashion database. We can use the provided BOE 4.0 Introscope dashboards to confirm that metrics are being collected for the agents that belong to the landscape that we want to trace. If you plan to trace some other workflow and are not sure which BI components are involved, you can refer to the official BI 4.x product tutorials for guidance. See below an example of the Webi Refresh workflow and the Introscope dashboards we need to check:
- Launch the CA Introscope Workstation using the URL http://IEMHOST:8081/workstation and logon to the Introscope Enterprise Manager (where IEMHOST is your CA APM Introscope Enterprise Manager hostname and 8081 is the HTTP port)
- Click Workstation, new Workstation
- Click the LENS button and sort by host, then multi-select all the hosts that belong to the BI landscape where you are performing the E2E trace and click APPLY. This will filter out all the un-necessary metrics and allow us to confirm that we are seeing metrics from the hosts involved in our workflow
- Next, click the dashboards drop-down and select the BOE 4.0 WEBI WAS dashboard
- Confirm that you are seeing metrics for the Webi Web Application dashboards
- Repeat steps 1-5 for each of the dashboards that are relevant to the workflow you are tracing. If you are unsure of which dashboards should be checked, refer to the BI4 RCA guide for detailed descriptions of each dashboard. Also, if you find that metrics are missing, refer back to the configuration guide and confirm that the Introscope Agents were configured correctly. For support on this topic, open an incident under component BI-BIP-DEP.
In this example, the dashboards to check include:
– Apache Tomcat Dashboard
– BOE 4.0 WEBI WAS
– BOE 4.0 PLATFORM SDK
– BOE 4.0 WEBI SERVER DETAIL
– BOE 4.0 WEBI SERVER OVERVIEW
Execute the E2E Trace
The SAP Client Plug-in is used to launch the browser and send the necessary header variables which will instruct Introscope to automatically record the Transaction Trace on the Introscope Enterprise Manager. The benefit of using the SAP Client Plug-in is that you do not need to manually start/stop the Transaction Trace tool inside Introscope Workstation which greatly simplifies the workflow. Download the SAP Client Plug-in according to your browser version and close all browsers before continuing with this section. Refer to the table below:
|VERSION||PLUG-IN DOWNLOAD||SAP NOTE|
|Internet Explorer 8, 9, 10||SAPClientPlugin_1.26_x86.zip||SAP NOTE 1435190|
|Internet Explorer 11||SAPClientPlugin.rar||SAP NOTE 1608231|
- First, on the client where you will be executing the trace, make sure that all browsers are closed (it’s also a good idea to check task manager and end task on all iexplore.exe processes)
- Browse to the location where you downloaded and extracted the SAP Client Plug-in and right click on plugin-starter-gui.exe and choose Run as administrator
- Confirm that Application is set to Microsoft Internet Explorer and click Launch
- After Internet Explorer launches, return to the SAP Client Plug-in and give the Business Transaction Name property a meaningful name and change the Next Step TraceLevel property to Medium. The Transaction Trace is programatically enabled on the Introscope Enterprise Manager when the SAP Passport is detected and the tracelog level is set to MEDIUM or HIGH)
- Next, queue up your browser to the place in the application where you want to begin tracing. In this simple example, we will trace a Web Intelligence refresh. When you are ready to begin the Transaction Trace, click the Start Transaction button on the SAP Client Plug-in then proceed with your workflow. Confirm the trace is working by checking the Sent Bytes / Recieved Bytes. When the trace is completed both should have values other than 0.
- When you have completed your workflow, click the Stop Transaction button then click ok on message “Please upload BusinessTransaction.xml manually”
Decode the SAP Passport with the BI Platform Support Tool
The key to fetching the E2E traces from Introscope Transaction Tracer is to decode the SAP Passport and query the Introscope Enterprise Manager for the historical trace records. To decode the SAP Passport from the BusinessTransaction.xml, you must use the new SAP Passport Decoder tool which is included in the SAP BI Platform Support Tool version 1.1.5 or higher. Download the tool from this wiki page
- Launch the SAP BI Platform Support Tool (v 1.1.5 or higher)
- From the menu, click Tools, then click SAP Passport Decoder
- Next click Browse, then browse to the location where your BusinessTransaction.xml is stored (this can be found in the /log/transactionName folder on the client where you extracted and ran the SAP Client Plug-in from). For example, C:\sapclientplugin\log\20140417_090959_Refresh Webi Document
- Notice that all of the information required to view the Transaction Trace has been extracted and decoded for you from the uploaded BusinessTransaction.xml
Query the E2E workflow in Introscope Transaction Tracer
Now that you have decoded the SAP Passport, you can now query for the Transaction Traces inside the Introscope Workstation
- Launch the Introscope Workstation
- Click on Workstation then Query Historical Events
- Next, click the drop-down box next to Time range and select Custom Range then set the time range according to the start and stop time in the SAP Passport Decoder tool then click ok. Make sure you specify a few minutes before and after the specified start/stop time to ensure you query for all the transactions that were part of your E2E workflow.
- Finally, copy the Introscope query from the SAP Passport Decoder tool into the Events Query box inside the Historical Query Viewer and click the GO button to execute the query
- The E2E Transaction Trace results are shown only for the workflow that you recorded using the SAP Client Plug-in
Analyze the E2E workflow in Introscope Transaction Tracer
- To begin the E2E analysis, first click the Timestamp column and make sure that the timestamps are sorted in ascending order then press CTRL-A or multi-select all of the transaction events in the Historical Query Viewer
- To view the most expensive methods and transactions for this recorded workflow, click the Summary View tab, then click the Call time (ms) column to sort descending. In this example, the processDPCommandsEx( ) method took the most processing time (2 calls at 1.7 seconds each)
- To analyze the E2E workflow in a heirachical callstack format to understand where a bottleneck might be, click the Tree View tab. Notice, in this view we can drill-down into the processDPCommandsEx( ) method to see which methods called by processDPCommandsEx( ) actually consumed the most time
- To view the E2E trace in a timeline format, click the Trace View tab. Notice, clicking on specifc methods reveals parameters passed into the method
Keep Going. I love all your posts.
Exceptional work as usual Toby. Keep them coming ➕
I get an error message when I try to load the XML file.
What could it be, I have kept strictly to the instructions?
It sounds like the SAP Client Plugin didn't create the BusinessTransaction.xml correctly. Is it a 0kb file? Maybe you could try another test generating the xml again using a simple browser workflow?
i think the file is written properly, in my case is the file 80kb. I also see the SAP Passport and X-CorrelationID. Upload the file works in the SolMan without problems, but not with the BI tool.
Can you email me the BusinessTransaction.xml ? firstname.lastname@example.org
Are you using the latest version of the BI Support tool version 1.1.9?
Thanks, I have sent you the file.
BI Support Tool Version is 1.1.9
Your XML was missing a transaction stop time.
I went ahead and generated the data for you and pasted below. I will add some exception handling for missing properties in the newest version of the BI tool (version 2.0)
2A54482A*4134444233303233314443333145443538304635414631433834353246304333*2A54482A and type:normal
cool, thanks. It works.
But introscope found only two traces, is there a possibility in the query filter, to put down the minimum transaction duration, so the introscope can select more traces? Like you can do it in "New Transaction Trace Session". 5000 is the default setting. About 5ms will be better.
Thanks for your post on the guide to setup introscope 9 for monitoring SAP BI 4.1.
I have completed the configuration and also am able to see the data in the dashboards. I have tried the following the steps in this post for analyzing E2E logs but all the time I see only one trace in the transaction tracer whereas when I create a new transaction trace session and repeat the workflow, I can see other traces. Would you be able to help me understand as to why the traces are not being shown when using the introscope query from the Business Transaction.xml?
Thanks & Regards
The query results you get in the Transaction Tracer depends on your workflow. With the BI Platform Support Tool's Passport Decoder, it is only possible to trace single step transactions. This means that the query will only return the traces from one step of a workflow (for example, a logon or a report view).
If you have a more complex workflow you need to click Next Step between each click of the workflow. The SAP Client Plug-in will generate a multi-step BusinessTransaction.xml. In order to filter on a multi-step transaction you need SAP Solution Manager E2E Trace Diagnostics tool.
Can you confirm, what is the workflow you are trying to record?
Thank you for your response and the clarification. Now am trying to trace only a single step - Refresh of a webi report. As in the below screenshot , I see only one trace from Tomcat Agent.
If you check in Introscope under the BI 4.0 WEBI SERVER DASHBOARD, are you seeing data there for all Webi nodes?
Thanks Toby, is there is document is decode function calls like "processDPcommands", "getpages", getmaps", "initinstance" etc.
Would be helpful to read traces.
Try this wiki:
Approaches to Error WIS 30270 troubleshooting - Business Intelligence (BusinessObjects) - SCN Wiki
answers to prompts to resume an interrupted refresh.
creates a document
executes the drill request
fetches binaries (such as images) and outputs (xls, pdf, ..) either from the session folder or from the cache
returns the filter bar definition
gets information on the document such as the properties of the document or the report specification
returns the list of reports in the document
generates outputs for clients or for export (pdf, xls…)
returns the list of prompts.
retrieves information related to the data sources.
creates the Webi session and logs in the user the very first time it opens a session with the server
opens the document from cache if available or from the repository
defines a custom sort on some objects.
commands on Data Providers to add a data source, refresh, …
adds, changes or deletes variables in the document.
returns the list of values for prompts
saves the document in the repository
changes the document properties such as “refresh on open”
Modifies the report (change tables, section, turn to chart, …)
Toby, Really great work!!
I am looking into similar tracing but for Design studio dashboards and Lumira, do you have any idea if the metrics for DS and Lumira are out in the market? Presently i am on WILY 10.1 and see no metrics enabled in the dashboards.
I neeed Wily for sap transaction tracing any idea where can I get it.