You may already be aware that Java 7 has been released and has a number of improvements over Java 6. If not, you can get the full story here or a couple of the biggest highlights from a BI Platform perspective on Idea Place. We'll discuss the "garbage first collector" mentioned in Idea Place specifically in this post.
Based on the popularity of this idea, and from general feedback from our customers and partners, SAP is currently targeting Java 7 support for Tomcat in a patch for BI 4 SP04. While this is a product future and the release schedule is subject to change, it represents a major move forward and should be cause for a lot of excitement. In this post I hope to convince you that there is significant value in preparing for Java 7 early if you run Tomcat 6 or 7 in order to move as quickly as possible when support is formally announced.
In order to do this, I came up with a few performance tests targeted at some of the exciting new features available in the BI 4 Feature Pack 3 release, namely the use of Exploration Views in the SAP BusinessObjects Explorer product. An Exploration View is a saved exploration on an information space that can be used to expose and explore a specific part of the data on the information space. Since this feature is expected to make a big splash with customers and in the Ecosystem, I decided to build one and see how Tomcat responded when hammered with a few hundred requests for this content.
I set up a test environment with the following specifications:
- 1 Windows 2008 R2 machine hosting the BIP 4.0 SP04 Server stack (CMS, FRS, Web Intelligence, etc) - 4 CPU/16 GB RAM
- 1 Windows 2008 R2 machine hosting the default BIP 4.0 SP04 Web Tier (Tomcat 6 + Java 6.0.24) and Explorer 4.0 SP04 servers - 4 CPU/8 GB RAM
- 1 Windows 2008 R2 machine hosting the BIP 4.0 SP04 Web Tier (Tomcat 7 + Java 7.0.4) and Explorer 4.0 SP04 servers - 4 CPU/8 GB RAM
- 1 Windows 2008 R2 machine hosting Apache JMeter 2.7 to execute the performance test
The 2 Web Tier machines were built to be exact replicas of one another ... the only difference being the version of Tomcat and the JDK that was in use. By using this configuration, I was able to execute a performance test against each machine, using the same BIP server tier, and measure the performance of each Tomcat+Java configuration. I'll share the very interesting results of these tests in just a few minutes.
Background
First, let me explain the performance test itself and the characteristics of my Exploration View. I recorded a test plan using the HTTP Proxy feature of JMeter that performed the following operations:
- Login to BI Launchpad
- Launch Explorer
- Navigate to Exploration View sets
- Open an Exploration View
- Logoff
The Exploration View is based on technical support data for some fictional customers that I used SAP Business Objects Data Services 4.1 and Sybase IQ 15.3 to generate. The characteristics are as follows:
- 5 charts
- 5000 rows in each chart
- 1 facet and 1 measure in each chart
The resulting Exploration View looks like this:
Test Execution
I then ran my JMeter test plan against the Tomcat 6/Java 6 and Tomcat 7/Java 7 environments to compare the performance of each. It is important to note that Tomcat 6 and Tomcat 7 performed almost identically when they both used Java 6 for the JVM. Java 7 is the real game-changer here, as I am about to demonstrate!
In order to take advantage of the new garbage first (GC1) collector in Java 7, we have to add at least one java option to the Tomcat startup script. On Windows, this can be done from the 'Java' tab in the Tomcat Configuration application. You can find an exhaustive list of the possible options for Java HotSpot VM usage here. Since my goal is to demonstrate the potential benefits from using Java 7, and not to write a detailed document on tuning garbage collection, I am only going to enable the GC1 collector by adding -XX:+UseG1GC to my Tomcat parameters.
The JMeter test plan was configured to emulate 50 distinct users, with a ramp-up period of 60 seconds, looping 5 times. This results in a total of 250 test plan executions, generating consistent load on the server for the entire duration of the test. I won't cover the configuration of the test here, but if there is interest I can do so in a subsequent post.
Results Analysis
Review of the Summary Reports for each test show the following details (click on the image above to see the numbers better):
- Using Java 6 an average transaction took ~29 seconds to complete
- Using Java 7 an average transaction took ~18 seconds to complete
- The standard deviation of a transaction was 40% less with Java 7, indicating a much more consistent experience
- The lengthiest transaction in both tests was launching Explorer and downloading the sizable demo videos that come with the product
I used JConsole to retrieve CPU and Memory statistics from the Tomcat JVM to determine why Java 7 results in better overall performance:
What we see for both CPU and Memory consumption is that the older JVM has many more peaks and valleys, resulting in a wider range of response times. Compare high water marks for memory usage of over 700 MB for Java 6, compared with just over 400 MB for Java 7. The Java 7 environment does sustain slightly higher overall CPU usage, but does a better job of keeping performance at a consistent level.
In summary, Java 7 brings much to be excited about, not least of all the proposition of a better experience for BI users and administrators alike. While this is only a test, and not a guarantee of improved performance in your environment, it is my intent to test more of the features present in Java 7 in hopes of convincing you all that it is an upgrade well worth pursuing.
17 Comments
