Elementary load testing of SAP Gateway oData services using cURL

[Former Member]

Introduction

Probably everyone understands the need to perform load testing of applications and services. Usually it comes down to how much time and money one can spend in doing that. This blog gives a quick and free approach. Although not very advanced, it can be used to simulate load easily.

Requirements

A fairly recent version of cURL is required. If you plan on load testing with HTTPS, your cURL needs to support HTTPS. The certificate of the accessed server needs to be trusted as well. With cuRL the latter can be a bit tricky depending on how cURL was compiled. In case cURL doesn’t have in-built support for certificates, you can retrieve the CAs from the cURL website and convert them to CRT format using the provided mk-ca-bundle. Name the output file curl-ca-bundle.crt and place in the cURL directory.

This blog was written while using Windows as a platform, it should be straight forward to map the used commands to your choice of Operating System, such as Linux.

Solution

First you need to solve how to get the CSRF token and use it for all POST requests. The solution is to invoke cURL with –i  to retrieve the header variables and –D and store them so that you can parse them, use

curl –s –i –H "Accept: application/json" -H "X-CSRF-Token: fetch" -D headers.txt -u user:password https://myurl/mylistservice

To parse the header variables use

findstr "x-csrf-token" headers.txt > token.txt

and

set /p TOKEN=<token.txt

to store the CSRF token in an environment variable called TOKEN. Now to use the CSRF token in POST requests, use –H “%TOKEN%” in addition to any other header variables you may want to set, e.g.

curl –s –H "Content-Type: application/json" -H "Accept: application/json" -H "%TOKEN%" –u user:password -X POST --data-binary @payload.txt https://myurl/mycreateservice

The payload (in JSON format) is stored in a text file called payload.txt.

CSRF token validation failed

If you followed instructions in this blog to the letter, you probably still couldn’t make it work. Any attempt to use POST would fail with a token validation error, even though you sent the CSRF token in the request. The most likely reason for this is that Gateway is generating a new token for each request, invalidating the previous one. The root cause is that Gateway doesn’t recognize the session. It’s not sufficient to pass the CSRF token together with Basic Authentication; you need to also pass the identifying session information. The session information is stored in Cookies. With cURL you’ll have to use –c parameter to store the sent Cookies and –b to send them back. In other words you would add –c cookies.txt –b cookies.txt to each cURL command. The commands used in our examples would become

curl –s –i –c cookies.txt –b cookies.txt –H "Accept: application/json" -H "X-CSRF-Token: fetch" -D headers.txt -u user:password https://myurl/mylistservice

curl –s –c cookies.txt –b cookies.txt –H "Content-Type: application/json" -H "Accept: application/json" -H "%TOKEN%" –u user:password -X POST --data-binary @payload.txt https://myurl/mycreateservice

Taking it up a notch

Now that you have your elementary script ready, you might ask yourself “how can I simulate load”, e.g. have multiple instances running. The quick solution is to run the script in as many directories as you want parallel instances. A more clever approach is to have cURL create and use files in one directory with a prefix/suffix tying it to a specific instance. I myself did the latter by using the parent process ID (of Command Prompt) as identifier, that way you can start as many scripts from the same directory as you want and as your systems can handle. To give you an idea how that would look, here is how the 2 commands in our example look in my script

curl -s -i -c _cookies_"%PPID%".txt -b _cookies_"%PPID%".txt -H "Accept: application/json" -H "X-CSRF-Token: fetch" -D _headers_"%PPID%".txt -u "%USR%":"%PWD%" https://myurl/mylistservice

curl -s -c _cookies_"%PPID%".txt -b _cookies_"%PPID%".txt -H "Content-Type: application/json" -H "Accept: application/json" -H "%TOKEN%" -u "%USR%":"%PWD%" -X POST --data-binary @payload.txt https://myurl/createservice

PPID environment variable stores the process ID of the parent process. Obviously user and password are stored in environment variables USR and PWD defined in the script. To use multiple users, you could either use a rotating (or random) list or use the multiple directory approach.

Other tricks

If you want that your script waits between individual commands (or loop iterations), you can use

SET /A SLEEP=%RANDOM% * 10 / 32768 + 5

timeout /t %SLEEP% /nobreak

to set wait time of 5-15 seconds. If you plan to implement the parent process ID approach you’ll soon find out that it’s surprisingly difficult to retrieve the process ID of the parent process, in Windows that is. I ended up writing a C# program and calling it from the script.

Conclusion

While the approach discussed in this blog is elementary, it does the job. While commercial alternatives provide more advanced and realistic load testing they usually come with a significant price tag, especially if you need to simulate tens, hundreds or even thousands of clients.