If I ask you, how to test the performance of an application, obvious answers would be to check the CPU Time, DB Time, Response time, Memory, Roll in – roll out time etc...
But, did anyone ever think that all the above parameters would depend on your System Health?
You’ll get the performance measurements on all the systems that you test on, but how can you be sure that these values are correct performance indicators of your application and not the overhead or an improvements occurred due to the system properties.
As an example if you are measuring the CPU time for benchmarking and your system is CPU Hyper-threading enabled, the measurements will differ from system to system depending upon the cores per CPU until the Hyper-threading is disabled on the system. If the performance measurements are being compared between two systems it is essentials to make sure that the CPU configurations are exactly same.
If possible performance systems should run on non-virtualized environments. Virtualization always causes a certain penalty in terms of CPU utilization. Also, make sure that file IO has no bottleneck. A Read Time of less than 10ms can be regarded as good. If a network attached storage (NAS) is used, determine the amount of hops between application server and filer by using trace route command, both should be as close as possible.
In modern CPUs power saving features are implemented to reduce the consumption of energy. In order to ensure reproducible CPU time, it is necessary to switch these features off. In SAP systems the Parameter “scaling governor” must be set to value ‘performance’ (by default the value is 'ondemand') and also make sure that irrelevant business functions are not activated in the system as this would also affect the performance.
Paging and swapping needs to be distinguished as well. Paging describes the common principle of moving data from memory to auxiliary storage and vice versa, e.g. during saving a document with an editor. Swapping is a sub-form of Paging and happens only if free memory in RAM is required or data isn't accessed for a certain time.
Permanent swapping activity during test indicates a wrong system memory sizing or a memory leak issue.
In SAP systems after running your first multi user performance tests, check in transaction ST02 the memory consumption peak values for all relevant areas like buffers, extended memory, and heap memory. If you observe buffer swaps, or values very close to the max Value defined, values need to be modified accordingly. Also, Make sure that an optimized profile based kernel is used.
There are a few small precautions that should be taken before starting the performance tests, ensure that no DB backups disturb your measurements.
A virus scanner can also slow down your system. So for performance systems we recommend de-activating an online virus scanner which checks every file for viruses, or schedule the virus scanners in such a way that it does not coincide with the performance measurements. Similarly make sure that even transports & batch jobs don't disturb your measurements.
We at STC Performance Testing Team take care of most of these and make sure that the Performance measurements have minimum overhead.
