SAP on IBM i: Reduce CPU utilization with 64K pages for the SAP kernel
Along with the development of the SAP kernel release 7.54, SAP has evaluated the usage of 64K pages instead of 4K pages for all the storage that is used by the SAP kernel and the work processes. What does that mean? Basically, the SAP kernel – like any other executable program – needs storage for its execution. In the PASE runtime environment, this is grouped into four sections:
- Initialized program data, uninitialized program data, heap segments (DATAPSIZE)
- Initial thread stack (STACKPSIZE)
- Main executable text (TEXTPSIZE)
- Shared memory allocated by the process (SHMPSIZE)
For each of these sections, you can select if this section is arranged in pages of 4 KB each or pages of 64 KB each. This can either be done at compile time through compiler options or at runtime through an environment variable LDR_CNTRL. To see the effective setting for the active processes of an SAP system, you can use the PASE shell command ps with the option -Z. For example, the ps command to show the settings for all processes running under user profile SIDADM would look like this:
ps -u sidadm -Z UID PID TTY TIME DPGSZ SPGSZ TPGSZ SHMPGSZ CMD 203 345534 - 1:18 4K 4K 4K 4K sapstartsrv 203 345602 - 0:00 4K 4K 4K 4K sapstart 203 345640 - 0:06 4K 4K 4K 4K msg_server 203 345641 - 0:38 4K 4K 4K 4K enserver 203 345803 - 0:58 4K 4K 4K 4K sapstartsrv 203 351245 - 0:00 4K 4K 4K 4K sapstart 203 351270 - 1:04 64K 64K 4K 4K disp+work 203 351271 - 0:00 4K 4K 4K 4K igswd_mt 203 351272 - 5:02 4K 4K 4K 4K igsmux_mt 203 351273 - 4:17 4K 4K 4K 4K igspw_mt 203 351274 - 4:17 4K 4K 4K 4K igspw_mt 203 351276 - 0:03 4K 4K 4K 4K gwrd 203 351277 - 0:09 4K 4K 4K 4K icman 203 351278 - 1:37 64K 64K 4K 4K disp+work 203 351279 - 1:05 64K 64K 4K 4K disp+work 203 351280 - 1:49 64K 64K 4K 4K disp+work 203 351281 - 1:44 64K 64K 4K 4K disp+work …
For the SAP work processes (executable disp+work), the program data and stack by default use 64K pages, while the executable text and shared memory by default use 4K pages. What is the difference between 4K pages and 64K pages? You can only allocate storage in a whole number of pages. If you need less storage than provided in a page, some storage gets allocated even though it is not needed. Obviously, chances are that the amount of allocated, unneeded storage is larger with 64K pages than with 4K pages. However, if you need a large amount of space allocated, then the number of pages is much smaller with 64K pages than with 4K pages, so the page administration requires less effort for the processor. In other words, 64K pages require less CPU resources but more temporary storage, while 4K pages require more CPU resources but less temporary storage.
If you want to reduce your CPU utilization and have enough main storage available, you can switch on the usage of 64K pages for the executable text and shared memory in SAP kernel releases 7.53 and 7.54. To avoid conflicts with other SAP components, make sure to have the kernel patched applied that came along with SAP Note 3283320 (SAP kernel 7.53: patch level 1112, SAP kernel 7.54: patch level 110). If the right kernel patch levels are installed, you can activate full 64K page support by adding a line similar to the following one to the start or instance profiles of all your instances, including the ASCS instance:
SETENV_03 = LDR_CNTRL=DATAPSIZE=64K@TEXTPSIZE=64K@STACKPSIZE=64K@SHMPSIZE=64K
Note that the parameter name SETENV_03 depends on other parameters in you profiles. In general, the parameter name is SETENV_<nn>, but you must first inspect your profiles for other parameters of that pattern to avoid duplicate names. We suggest that you replace <nn> with the next available number. For example, if your profiles already use the names SETENV_00, SETENV_01 and SETENV_02, the next available name would be SETENV_03, as in the example above. If only SETENV_00 was used up to now, you would select a parameter name SETENV_01. Obviously, profile lines starting with the comment character ‘#’ need to be skipped in that process. The setting takes effect at the next restart of the SAP system.
When testing the full 64K pages settings with the SAP Standard SD benchmark at about 70% overall CPU utilization, we noticed a reduction of the CPU utilization of about 5% and a reduction of the average response times in the dialog work processes of about 10%. If your overall CPU utilization is higher than 70%, you may see a higher reduction. If the database time is a significant factor in your overall response time, you may see less reduction as the 64K page size only affects storage used by the SAP kernel.