Skip to Content

Introduction

During the course of a recent project for a customer, a fairly complex Web Dynpro for ABAP application was developed. The WDA was a single screen application, with several popup screens, each implemented as a separate WDA. The WDA displays the operations of an existing work order and lists a set of activities per work order operation. For each activity, the user can create a number of different types of notifications. It was estimated that the application would have about 1800 concurrent users when released to production. This raised some concerns about  possible performance issues with respect to the application in particular and the SAP Internet Communication Framework in general. The customer had little previous experience with these types of custom Web Dynpro applications, and there had been some experience with applications on other technologies failing under heavy load at go-live. When the customer asked whether the application might break under load, it was not really possible for the project members to deny this possibility with 100% certainty. Based on our experience with Web Dynpro and ICF we were fairy confident that the application and infrastructure would hold, but to better support our gut feeling with facts, we decided to undertake a load test of the application.

Performance testing versus load testing

The terms performance testing and load testing are often used interchangeably and the difference between the two is not well defined. In this article, the following definition is suggested:

Web load testing is:

  • similar to, but not synonymous with performance testing
  • concerned with the volume of traffic your website (or application) can handle
  • not intended to break the system
  • viewing the system from the user perspective
  • associated with black box testing

Web performance testing is:

  • a super-set of load testing
  • concerned with speed and efficiency of various components of the web application
  • useful with only one user and/or one transaction
  • viewing the system from the architecture perspective (behind the server side curtain)

In our case, our primary concern was ensuring that the application would not break under heavy load, that is, become totally unresponsive. Within reasonable limits, we were less concerned with the user experienced performance of the application. Thus, load testing was our primary objective, but our findings would also be useful for performance considerations.

A note on performance testing

When testing the performance of an application under heavy load, three components are actually being tested:

  • The application
  • The application framework – in our case, the SAP web dynpro framework and the SAP Internet Communication Framework (ICF)
  • The network infrastructure

Of these three components, we have little influence over the framework and the network. Improving performance would mainly be accomplished through fine tuning of the application. Possibly, some measures might have been taken to improve network and framework performance as well, but this never turned out to be necessary.

Choosing a tool for the job

There are many tools available for load testing of web applications, both commercial and free. Several of the available tools were considered for our task. The following is a list of other tools that were considered and/or tried:

  • Microsoft Visual Studio Ultimate – comprehensive, very expensive. No support for cookies without programming.
  • HP Load runner – expensive – has been used previously by customer, but was discouraged based on high cost
  • Selenium – not really a load testing tool, aimed at functional testing
  • Winshuttle – a data loading tool for SAP, but does not support Web Dynpro
  • Apache JMeter – open source, free, Java based, very configurable

JMeter was recommended to us at an early stage based on previous experience at customer site, and it soon became apparent that JMeter was the best option in terms of functionality, even without considering price. Our main requirement was being able to simulate the load of many simultaneous users, and JMeter met this requirement, at no cost.

Getting JMeter

Apache JMeter is a free and open source tool for performance testing of web applications. It is available here

Other useful tools

In addition to JMeter some other tools are also necessary or useful for developing performance tests with JMeter. You will need:

  • A text editor with support for regex searches – recommendation: Sublime Text
  • A tool for inspecting HTTP traffic – recommendation Fiddler
  • A modern browser – recommendation: Chrome

How JMeter works

JMeter differs from many other testing tools in that JMeter only works at the HTTP level, and not on the presentation level where HTML pages are actually rendered. JMeter is not a browser. JMeter will act as a browser against the web application server (such as the SAP ICF which serves web dynpro applications) in the sense that JMeter sends HTTP requests and receives web pages in the form of HTTP responses from the server hosting the web application. JMeter differs from a browser in the sense that JMeter does not execute the Javascript code in the received HTML pages and JMeter also does not render the HTML of the web pages to screen.

When running JMeter, typically many “threads” are executed in parallel – each thread represents one user session. Thus JMeter can simulate many users simultaneously accessing a web application. Spinning of thousands of threads on the workstation JMeter is running on would possibly cause the workstation to run out of memory, therefore JMeter also supports running test scenarios distributed across several workstations. In our testing, we ran JMeter with 400 threads each on 2 workstations to simulate the load of 800 concurrent users.

How Web Dynpro applications work

A Web Dynpro application is a web application hosted on the SAP server. When the browser accesses the URL of a web dynpro application, Javascript, HTML, CSS and resources such as images are downloaded to the browser and rendered on screen. When the user interacts with the application by for instance clicking a button the following happens:

  1. A HTTP POST request is sent to the SAP server with information on what type of user action that occurred (what type of action was performed on wich UI element)
  2. The Web Dynpro framework communicates the user action to our Web Dynpro application code
  3. Our Web Dynpro code typically enters a new state based on the user action (for instance, a radio button is set)
  4. The Web Dynpro framework sends a HTTP response to the browser containing the new view in its updated state, and the new application state is rendered on screen

It is important to note that with Web Dynpro applications, every user action results in a new HTTP POST request being sent to the server which will than send an HTTP response with an updated application state to the browser. Many modern web applications will react to user actions via Javascript code on the client (browser) and update the screen without doing a round trip to the server. With Web Dynpro applications every user action results in a round trip to the server and a redraw of the entire screen (disregarding any optimizing on the browser). This behavior is a good fit for JMeter since JMeter only works on the HTTP layer, inspecting HTTP requests. JMeter is not able to execute or react to Javascript code being executed on the client.

Setting up a test scenario

Start JMeter, and create a new “Thread group” by right clicking on “Test plan”. A thread group is a collection of HTTP request that are passed to the web server hosting the web application under test. After we have added a series of request to the thread group, we can run the thread group with a large number of threads (users) to simulate multiple users accessing an application simultaneously. For a thread group we can also add post- and preprocessors that manipulate each request, monitoring and statistics, logical controllers and so on.

Recording as user session

JMeter can record HTTP traffic and store all the requests that are passed to the web server. The requests are stored in a thread group and can then be replayed from JMeter to simulate a user accessing the web server. To be able to record a user session JMeter must act as a proxy server – if a browser is used to create HTTP traffic, which is the most common case, the browser must be configured to use JMeter as a proxy. Traffic from the browser is then routed to JMeter, and JMeter passes the traffic along to the web server, receives traffic from the web server and passes it back to the browser. By acting as an intermediary, JMeter can record all the traffic.

Setup a recording

  1. Right click on the thread group and select “Add – Logic controller – Recording Controller”. Our recording will be stored in the new node “Recording controller”
  2. Right click on “WorkBench”, select “Add – Non-test elements – HTTP Proxy server”
  3. Right click on the “HTTP Proxy Server” node and select “Add – Listener – View Results Tree”. The listener will store every information and response, and we need this information later on
  4. Click on the resulting proxy server node and make a note of the Port number (typically “8080”). Also make sure to choose “Use recording controller” as the value for “Target Controller”
  5. In the settings for your browser, set JMeter as proxy by entering the values for the server address “127.0.0.1” and port “8080”

Your setup should look like this:

JMeter - recording with listener.png

Recording

Start the recording by clicking “Start” on the HTTP Proxy screen. Traffic from your browser will now pass through JMeter and be recorded. Recorded traffic will be stored in the “Recording controller” node. The result of recording a brief web dynpro session will look like this:

JMeter - Recording controller result.png

As can be seen from the screen shot, most of the HTTP traffic comes from the browser downloading resources such as images, CSS and Javascript. When we initially access the application, and whenever we access a new screen, lots of these resources will be downloaded by the browser. The initial downloading of resources will contribute to the load on the system, but as the resources are cached by the browser (and pn the server) they will have less impact on the totel load. In my experience, the contribution of these resources on the overall load was negligible, so for the sake of simplicity we can filter those files from being recorded by JMeter. We do this by adding a regular expression to “URL Patterns to exclude” for each file we want to ignore:

JMeter - proxy exclude filter.png

Recording again, we now collect fewer and more interesting requests. We are only interested in the requests that are highlighted – the requests that contain the Web Dynpro application name – the other request can be deleted:

JMeter - recording with filter.png

Running the scenario

After having successfully recorded a user session with the WDA, you can run the test plan by clicking the green play icon (“Run”). When the test plan is run JMeter will fire the HTTP requests in the recording controller to the server and receive the HTTP responses. For a simple web application with no authentication this would work fine, but with a SAP server it won’t. The SAP server expects a user session to be authenticated and all HTTP requests to have two unique IDs which identify this particular user session. The authentication scheme in your SAP landscape may differ, but for our SAP system the following process is involved:

  1. When a user first accesses the corporate intranet, a SAP single sign on cookie called MYSAOSSO2 is issued for the same domain as the SAP WDAs are hosted (companyname.com). This cookie is valid for one day
  2. When a client (browser) accesses the URL of the WDA by sending a HTTP GET request, the MYSAOSSO2 cookie is passed along by the browser
  3. The SAP server responds to the GET request and sends two tokens in the HTTP response body, sap-contextid and sap-wd-secure-id. The SAP server expects these tokens to be included in all subsequent requests from the client to uniquely identify the user session

So for JMeter to successfully communicate with SAP the following is required:

  • A single sign on cookie must a included with all request
  • The query parameter sap-contextid must be passed with all request after the initial GET request
  • The parameter sap-wd-secure-id must be included in the body of each request after the initial GET request

Luckily, JMeter is capable of handling all this. The “HTTP Cookie Manager” will (unsurprisingly) handle cookies, and we can use “Regular Expression Extractors” to extract the sap-contextid and sap-wd-secure-id tokens from the first request, store them in variables and add them to subsequent request.

Working with the recording

Our new, cleaned up recording looks like this:

JMeter - clean recording.png

Adding Regular expression extractors

Note that the two first requests are HTTP GET requests that initially access the application on the server. The subsequent requests are HTTP POST request, one for each user command. In the first of the GET request (the second GET request can be deleted from JMeter as it is not needed for running the test scenario), two tokens are passed to the client in the HTTP response. These tokens are used in all subsequent communication between the client and the server to authenticate the user session. We can find the values of these tokens by inspecting the request in the “View Results Tree”:

JMeter - request in listener.png

JMeter - tokens in HTTP response.png

The sap-contextid token is appended to every request as a query parameter (at the end of the URL) while the sap-wd-secure-id token is passed in the body of the request. The value of these tokens will differ with every user session, so to be able to playback our recording more than once we must dynamically extract the value of the tokens and append them to our requests. JMeter makes it possible to extract data from HTTP responses using the “Regular expression extractor”.

Add the extractor by right clicking on the first request in the recording controller and selecting “Add – Post processors – Regular expression extractor”. We need to add one extractor for each parameter. We also will add a HTTP Cookie Manager and a HTTP Header Manager – the purpose of which is explained below – giving us the following configuration:

JMeter - added extractors and header manager.png

The regular expression extractor is a post processing controller – it is triggered after a HTTP response has been received by JMeter. The extractor will extract a value from the response that matches the regular expression and store the value in a JMeter variable. The variable can then be used in subsequent requests and controllers. We must define the following parameter for the extractor:

  • Reference name – name of the variable in which to store the retrieved value
  • Regular expression – the regular expression
  • Template – the number of the regular expression group to be stored in the variable
  • Match no. – if there are more than one match, the number of the match to store in the variable

In our case we will store the contextid and secure-id into two variables and add them as parameters to all requests. For the secure-id extractor we will enter the following values:

  • Reference name – sap-wd-secure-id
  • Regular expression – name=”sap-wd-secure-id” value=”(.+?)”
  • Template – $1$
  • Match no. – 1

For the contextid extractor we will enter the following values:

  • Reference name – sap-contextid
  • Regular expression – (SID.+?NEW)
  • Template – $1$
  • Match no. – 1

You may have to tweak the regular expression to extract the values correctly – regular expressions are a pain in the neck, but there are some online tools that help you test your expressions. For sap-contextid, note that I selected “Body (unescaped)” to get the correct, unescaped value for the context id. When testing your expressions in JMeter, add a “View results tree” node to the top node, run the scenario, and inspect the requests and responses in the listener.

JMeter - extract sap-contextid.png

JMeter - extract sap-wd-secure-id.png

HTTP Cookie manager

The cookie manager allows JMeter to send and receive cookies to and from the server in the same way a browser does. Cookies are generally used for authentication of the client against the server, and depending on the authorization scheme on your SAP server a cookie will be used to authenticate the user sessions together with the secure-id and contextid tokens. In our system landscape, a SAP single sign on cookie is obtained when the browser accesses the corporate intranet the first time. This SSO cookie is then used as authentication when accessing any SAP server. JMeter is unable to obtain this cookie in the same manner as Internet Explorer is. But since the cookie is valid for an entire day, we can access the intranet using Internet Explorer, and then we can find the value of the cookie and copy it into the JMeter cookie manager manually. Use Fidller or “Developer tools” in IE to find the cookie; press “F12”, select “Cache – View cookie information”. In our system the cookie is called “MYSAPSSO2”:

JMeter - MYSAPSSO2 cookie.png

In the cookie manager in JMeter I will create a “User-Defined” cookie, called “MYSAPSSO2”, and copy the domain of the cookie and the value of the cookie from Internet Explorer:

JMeter - HTTP cookie manager.png

HTTP Header Manager

The “HTTP Header Manager” is used to add headers to all HTTP request that are sent from JMeter to the server. We need to add a header called “User-Agent” with the value Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0) to the initial request to the server to trick the SAP server into believing that it is communicating with a browser, otherwise the SAP server will send a message that our browser (JMeter) is not supported:

JMeter - HTTP header manager.png

Modifying requests to include contextid and secure-id

Having successfully extracted the contextid and secure-id into variables, we now must modify all our requests to pass these tokens. A typical request will look like this:

JMeter - request with variables.png

${sap-contextid} and ${sap-wd-secure-id} are references to the variables.

You can determine whether your requests are successfully authenticated by adding a “View results tree” listener to the test plan node, running the test plan, and inspecting the results in the listener. Select a request in the listener, select the “Response data” tab, and select HTML from the drop-down menu at the bottom left. This will make JMeter display a crude rendering of your web dynpro application screen. If unsuccessful, you will see some some kind of SAP error message instead, because your request was not authenticated.

jMeter - request response as HTML.png

Looking at the request in more detail

The initial HTTP GET request establishes a connection to the SAP server and receives the user session tokens. After this point, every user action on the client will generate a HTTP POST requests with information on the user action, and the server responds with an updated screen in the HTTP response. Information on the user action is contained in the parameter SAPEVENTQUEUE:

JMeter - SAPEVENTQUEUE.png

To be able to manipulate the state of the web dynpro application this parameter must be passed. There are 2 ways this can be done:

  1. Either record a user session using JMeter HTTP proxy and update each request with the variables for contextid and secure-id afterwards, or
  2. Interact with the web dynpro application while a tool such as Fiddler is running. In JMeter, simply add request manually to the test plan, and use values found in Fiddler for the SAPEVENTQUEUE parameter

Below is a typical recording from in Fiddler:

JMeter - request in Fiddler.png

I preferred method number 2 as it gave more control over the development of the test script in JMeter. In JMeter, I can create a new request (or more often, copy an exiting one) and copy value of the parameter SAPEVENTQUEUE from Fiddler and paste it into JMeter.

Stable IDs

Every UI element in the web dynpro view is assigned an ID when the page is rendered in the browser.

JMeter - HTML source - non stable ID.png

For example, the <td> element above contains a radio button element, and the ID of this element is “WD66”. This ID is sent in in the SAPEVENTQUEUE parameter in a request to the browser when a user operates on the element. In our test script in JMeter this ID is stored as part of the request. However, it turns out that the ID of the UI element can change between each run of the application. This means that the JMeter test script can fail because it uses an incorrect ID for the UI element. Luckily, SAP has provided a solution to this problem. Setting the URL parameter sap-wd-stableids=X will cause every UI element to be rendered with a stable and more meaningful ID, as shown below for the same radio button element:

JMeter - HTML source - stable ID.png

So make sure to set this parameter for every request to ensure that the test script will not break.

Running the script

Below is a screenshot of my script after I have tidied it up and given each request a sensible name that describes to the user action in the application:

JMeter - running script.png

In the log on the right side is the result from running my script — the log shows no errors. Note that the log would also show no error even if JMeter was unable to authenticate against the SAP server, because in that case the SAP server would return a HTTP response with status code 200 containing an error message. JMeter will only show status codes different from 200 as errors. So to determine if the script ran successfully, either the responses must be investigated in JMeter or the effects of running the script should be checked in SAP.

Simulating many users

Still we have only run the script in a single thread, simulating just one user. The whole point of using JMeter is to simulate many users. This can be accomplished by selecting the thread group “Main”, and setting the value of “Number of threads (users)”:

JMeter - set number of threads.png

Here I have set the number of users to 10. A ramp-up time of 1 means that the threads will be active within 1 second. It is also possible to run the script in a loop to average measurement results.

Measuring performance

JMeter provides several types of listeners for collecting performance measurement data. The most useful one is the “Summary report” which collects key data such as average, minimum, maximum and median response time for requests. These numbers correspond to the time the user has to wait for an operation in the application to complete and become visible on screen. Below are three different measurements from running the script 200 times with one user, for a total of 1400 requests:

JMeter - aggregate graph.png

JMeter - summary report.png

JMeter - graph results.png

Every type of listener will incur a penalty on the performance, so fewer and less detailed listeners are recommended. Graphing performance metrics in real time in JMeter is fun, but it’s more useful to use the summary report and export the data to Excel afterwards to analyze and graph the numbers.

Conclusion

The main objective of our load test was to prove that the application would not break under heavy load after going live. It is not really possible to positively prove this, because the famous “unknown unknowns” can contribute to bring the application down somehow. Load testing with JMeter did however give us great confidence that our application would not break, and we were able build confidence with the customer as well based on the numbers and measurements from JMeter. We also gained confidence the SAP ICF framework – we never got close to bringing ICF to its knees (as far as we know), and ICF appears to be very robust, which alsobodes well for its new role as the underlying platform for UI5 applications.

Load testing should not be mandatory for all new Web Dynpro applications – but for critical applications with a great number of concurent users it makes sense. Load testing with JMeter will also reveal information on performance issues in an application that would otherwise be hard to detect. Even if that was of secondary importance in our project, through visualizing the measurement from JMeter we we learned a lot about where there was room for improving the user perceived performance of our application.

To report this post you need to login first.

12 Comments

You must be Logged on to comment or reply to a post.

  1. Andy Silvey

    Hi Frank,

    this is a very nice and useful blog, thanks.

    Only from curiosity it would be interesting as an addendum to go into the definitions of other forms of testing for example sink testing etc,

    Thanks again,

    Andy.

    (0) 
    1. Frank Stødle Post author

      Thanks Andy!

      Yes, it would be interesting with a fuller definition of the different forms of testing. I spent little time researching this, but I was a bit surprised at not finding any “canonical” definitions on the different terms.

      My blog is more of a “hands-on”. I had some real trouble getting JMeter to work with Web Dynpro because of the authentication issues, so my blog is mostly about addessing these issues.

      (0) 
      1. Andy Silvey

        Hi Frank,

        yes, the blog is perfect, I’ve saved it as a pdf incase I’m ever asked to arrange load testing 🙂

        Thanks again,

        Andy.

        (0) 
  2. Frank Ruggaber

    Hi Frank,

    thanks for this great blog. We’re using JMeter many years for loadtesting our Web Dynpro, BSP and portal applications. Our conclusion after all: even if it takes some times to get around this http stuff – compared to other products available on the market – it is a perfect tool for loadtests.

    Two things to add to your blog:

    • the simulation of many users can also be done with different user logins, if you use the “users.xml” feature of JMeter.
    • in a many user scenario (>100 parallel users) we added the “constant throughput timer” element in our test plans, to simulate a more realistic user behaviour.

    Regards

    Frank

    (0) 
    1. Frank Stødle Post author

      Thanks Frank,

      it’s good to learn that you too have positive experiences using JMeter for WDAs, that means I have not completely missed the mark. I found that once I had figured out the hard bits it was relatively quick to put together a test scenario in JMeter, and all together I am very happy with the tool.

      I agree, using many different logins would make the test even more realistic. In my actual testing I also randomized request times using the Gaussian timer, but for the sake of simplicity I did not include those details in the blog

      (0) 
  3. Former Member

    Useful article but I think checking Body (unescaped) in Regular Expression Extractor may cause OutOfMemory error during load testing. Also I would recommend exclude all listeners and use only Simple Data Writer in csv format. As you know JMeter written in Java and that’s why needs a lot of memory. For this reason for load testing better tune up your JMeter script.

    (0) 
  4. Former Member

    Hi, Frank

    Thanks for sharing your knowledge about WDA testing. I am using HP loadrunner though and found that each HTTP post results in a new user session created in ECC. I was monitoring with SM04/AL08. Is there a way to prevent this?

    That means, when I run my script with 5 virtual users, I can end up having 200+ user sessions, depending on the length of the script.

    Thanks,

    Jonathan.

    (0) 
    1. Frank Stødle Post author

      Hi Jonathan,

      sorry for my late response, I’ve been on holiday.

      Are you including the sap-wd-secure-id and sap-contextid parameters in all HTTP requests?

      (0) 
  5. Former Member

    Hi Frank,

    Thanks for hits useful article. Currently I’m creating jMeter test scripts for various WD applications and have problems getting stable IDs.

    One test user gets views with stable IDs, while another user running the identical test script doesn’t. Every HTTP call has sap-wd-stableids in the URL query string and also the order in which the users are running the script doesn’t matter. Do you have any idea? Do SAP users have to specifically be enabled to use stable IDs?

    Thanks in advance!

    Remco

    (0) 
  6. Former Member

    Hi Frank,

    Thanks for sharing this useful information. We have a similar scenario in one of the projects we are doing currently and planning to use JMeter. I need to do the following.

    1. There is a WebDynpro screen where user can browse a file from local drive and load the content of that file in DB tables by pressing the submit button.

    2. Users are segregated by various roles and they can submit data only for their respective departments.

    With JMeter, Is it possible to simulate this scenario with 50 different test users (for different departments), loading different files at the same time? For each user we need to have different Test UserIDs/Password and different files.

    Thanks in advance.

    Sachin

    (0) 

Leave a Reply