Performance Testing in BI 4.1

I’ve recently spent a bunch of time getting to know BI 4.1, and part of that process is taking it for the proverbial test drive in terms of performance.  To do this, I always use Apache JMeter since it is free and a great overall product.  I’ve been documenting the process for designing performance and regression tests on BI 4.0 on the SCN Wiki for some time, but with the popularity of this topic, along with the added exposure gained in my recent ASUG webcast on automated regression testing, I haven’t made it as far as I would like in the process.

Due to challenges and other commitments, I just got around to creating a test plan for BI 4.1.  The test plan invokes 3 different BI clients … Web Intelligence, Explorer, and Analysis for OLAP, and all get backend data from HANA.  I’ve uploaded the JMX code for this test plan on the wiki:

http://wiki.sdn.sap.com/wiki/display/BOBJ/4.1+Mixed+Test+Plan

And this represents a new process for me that should be sustainable in the long-term.  It was taking such a time commitment to exhaustively document every step in test plan creation that I found myself with dozens of functional test plans, but only a few of which were documented for use.  Therefore, I propose to put all of my test plans on the wiki as source code.  You can simply copy and paste the code into a file that you save with a .jmx extension, and then you’ll be able to open it directly in JMeter.  I will use this space as a means to blog about the test plans, and explain the key components you’ll need to customize to use them in your own environment.  I’ll look forward to your feedback on whether or not this process works and can fine tune as necessary.

You can familiarize yourself with JMeter and create your first test plan (exhaustively documented) here:

http://wiki.sdn.sap.com/wiki/display/BOBJ/Creating+your+First+Test+Plan

BI 4.1 Mixed Test Plan

First, in order to open this test plan, which uses JMeter plugins to instrument system resources such as CPU and Memory, you must follow the instructions found here:

http://scn.sap.com/people/james.rapp/blog/2011/07/15/integration-of-server-resource-monitoring-with-jmeter-plugins

Once you have added these plugins to the JMeter lib/ext directory you can open the test plan and see this goodness!

The test plan is broken down as follows:

1. Thread Group configured (by default) for 100 users and a 300 second (5 minute) ramp-up time.  This means that after 5 minutes, 100 users will actively be performing BI workflows on your server.

2. Each user logs on to the environment, as controlled by the requests in the Logon transaction controller.

3. Each user opens and refreshes a Web Intelligence report.

4. Each user opens a previously saved Exploration View Set.

5. Each user creates a new Analysis Workspace, selecting a connection, adding a dimension, and inserting a chart.

6. Each user logs off of the system.

Unique users are leveraged for each thread and that process is configured in the BI Users CSV config.  At the end of each transaction controller is a Uniform Random Timer that causes a delay between 100 ms and 2000 ms (2 seconds) between each request.  This emulates a business user performing some analysis and keeps the requests from flooding the system.  The timer duration is not included in the results so we get a pure view of how long each operation takes.

Breaking it Down

I cover the Logon/Logoff methods in the wiki page I provided above, but logging onto BI 4.1 requires one slight alteration from BI 4.0.

BI 4.1 uses a Java Server Face to process the BI Platform logon so it is necessary to pass a parameter called com.sun.faces.VIEW to the logon.faces request.  I struggled with this initially and discovered that if you don’t pass the value the logon page simply gets re-postedand you never get a session!

Here is what the regular expression extractor should look like:

We need to pass this value into the following logon.faces request:

From here I will just talk about the individual BI clients.  Web Intelligence gets to go first by virtue of its position in the test plan:

Web Intelligence

Regular expression extractors are attached to 3 samplers in the Web Intelligence workflow.  The values extracted are:

1. The SI_ID of the user’s Favorites Folder

2. The sEntry representing the Webi document state when the report is first opened.

3. A second sEntry value representing the Webi document state when the report is refreshed.

Once those values are extracted they should be attached to the appropriate Webi samplers as JMeter variables.  JMeter variables take the form ${fav}, ${sEntry}, etc.

Since JMeter test plans are plain text it is easy to replace static strings such as an sEntry (sEntry might take the format we0001000078432c722f78)

Refer to this new SCN Document for details on how to do the text replacement!

The result of the Web Intelligence workflow looks like this:

In the interest of length of this article, I’ll wrap up this post for now and follow up with another post next week to articulate the Explorer and Analysis OLAP workflows.  I’m really excited to be performance testing with BI 4.1 … my initial tests show it performs really well and I’ll look forward to sharing more of these details with you.