Java Memory Sizing procedure (Pragmatic approach)

How to determine the Java memory requirements for customer load?  How to define optimal number of Java Servers to handle the load?

***

Introduction

Often the required java memory is calculated by applying some “guru-defined” ratio (e.g.”4GB per vCPU” or “4 GB memory per 1000 SAPS”) or is determined in experimental way (e.g. load test) for a concrete load profile, or even it is not calculated at all but later, in production, changes to JVM parameters are applied as reaction to already experienced java memory problems.

This blog suggests a pragmatic approach to calculate the required memory for java applications (i.e. JVM settings and number of Java server instances) based on defined application memory consumption metrics.

Be aware that Java Memory Sizing is only relevant to memory-leaks-free applications – the java memory sizing approach helps to avoid only memory shortage situations, which are not leaks.

 

Short overview on JVM Memory Management

It is typical for Java applications that
– most of the created objects become unused very soon
– many objects and classes, once instantiated, remain in memory for a long time

To serve to this reality, Java Virtual Machine implements a concept of separate memory areas for different ages and types of objects/classes: young generation, old generation and permanent generation.  The Java Virtual Machine (JVM) manages objects de-allocation automatically (i.e. performs the so-called “garbage collection”) using varies garbage collection algorithms, which improved over the years.

Before you proceed, if you are not familiar with the garbage collection mechanisms, please check https://www.oracle.com/webfolder/technetwork/tutorials/obe/java/gc01/index.html.

 

The garbage collection duration depends on the garbage collection algorithm, the size of the young, tenured and permanent generation area, the CPU processing speed, some specific influencing factors like implementation of finalize() method of java objects, and so on. With bigger heaps or with less powerful CPUs, the duration of both small and major garbage collection is longer. Memory paging increases the duration of the garbage collection to unacceptably long execution times, and thus it is absolutely essential to have the complete java heap inside the main memory of the machine.

 

ℹ Example for duration of small garbage collection

On CPU @ 2.66GHz , -Xms=4096M –Xmx=4096M –XX:PermSize=512M –XX:MaxPermSize=512M –XX:NewSize=820M –XX:MaxNewSize=820M, the minor (small) garbage collection takes about 0.8-0.9 seconds.

 

ℹ Example for duration of full garbage collection:

On CPU @ 2.66GHz, -Xms=4096M –Xmx=4096M –XX:PermSize=512M –XX:MaxPermSize=512M –XX:NewSize=820M –XX:MaxNewSize=820M, the major (full) garbage collection takes about 9-10 seconds.

With the major (full) garbage collection the entire heap must be examined for unnecessary objects, that is why it is much slower compared to small garbage collection.

 

➕ SAP recommends to use the JVM parameters -XX:+UseConcMarkSweepGC and -XX:+UseParNewGC to enable parallel garbage collection algorithms.

 

➕ SAP recommends that the values for JVM parameters pairs -Xms and –Xmx, –XX:PermSize and –XX:MaxPermSize, –XX:NewSize and –XX:MaxNewSize should be set to same value – in this way heap areas will not resize at runtime, because the resize is associated with worst performance.

 

Usually customers choose heap sizes (-Xms and –Xmx) between 2 GB and 8 GB per Java Server node. Larger heaps are not very common, though possible.

Native code area

The JVM implementation needs space outside defined heap, let’s name it “Native code area”. Consider at maximum 1 GB per JVM for it.

 

 

Short overview on basic memory consumption metrics

in SAP Java Application Server environment

From java application perspective there are several memory metrics which determine its memory consumption profile.

Framework Space [MB]

The so called Framework Space is filled with objects created during initialization of java server services, libraries, applications, etc. and objects of shared caches, pools, etc. Such objects typically live for the entire life time of the java virtual machine and therefore are always promoted to the tenured generation. The corresponding to those objects classes and methods, defined as Perm Framework Space, stay in permanent generation for the complete live-time of java server.

ℹ The measurement for Framework Space and Perm Framework Space is done via analysis of heap dumps, taken after warm-up (execution with different test users of the main functional scenarios) of the sized application. It is important that at the time when the heap dump is taken the users are no longer active on the server node – they should be either logged out or their sessions should have expired.

Use Memory Analyzer tool. https://help.sap.com/saphelp_em900/helpdata/en/cc/2c25047b0049769e60cde988dbc715/frameset.htm

User Session Space [MB per session]

The so called Session Space is filled with objects related to the user session and temporary caches which are removed if the user session is destroyed due to log out, or timeout. A user is connected to the system for at least several minutes – the SessionSpace objects usually live long enough to be promoted to tenured generation and from tenured generation area they could be removed only with major (full) garbage collection.

ℹ The memory measurement for user Session Space and user Perm Session Space is usually done with analysis of a heap dump, taken while multiple users are logged in to the system and are executing different dialog steps of the important functional scenarios of the sized application.

Use Memory Analyzer tool.

https://help.sap.com/saphelp_em900/helpdata/en/cc/2c25047b0049769e60cde988dbc715/frameset.htm

Processing Space [MB per dialog step]

The Processing Space is filled with short living objects which typically are ready to be immediately garbage collected directly from young generation space after processing of a request or task is completed.

ℹ The memory measurement for Processing Space is based on the total size of all object allocations done in the thread, or multiple threads, which are executing the user request. The Processing Space includes not only the objects which are temporary allocated and garbage collected for processing the user request but also those objects which will become part of caches or session space, i.e. will be promoted to tenured generation after small garbage collection.

Use SAP JVM Profiler tool.

https://help.sap.com/docs/btp/sap-business-technology-platform/profiling-application-running-on-sap-jvm

 

Goal of Java Memory Sizing

The goal of java memory sizing is to minimize the frequency and duration of garbage collection, i.e. to determine with which (application specific) size of
– young generation, a minimum amount of objects would be copied into the tenured generation
– tenured generation, a minimum number of full garbage collections would be executed
– permanent generation, a minimum number of full garbage collections would be executed

If best conditions for minimum duration of GC are in place via selection of the best optimized garbage collection algorithm, fast CPU model and adequate sizes of generation areas, we can effectively define “target” garbage collection intervals to ensure that garbage collection does not happen too frequently.

The frequency, defined as interval between two successive garbage collection (GC) occurrences, is a required input parameter for memory sizing.

Let‘s define Isgc and Ifgc

· Isgc = interval of small garbage collection (in seconds), for example 3 seconds

It is recommended that Isgc is longer compared to average response time for particular end user request. For example, if the average response time per request is 1-2 seconds then target interval of 3 seconds is good enough. If the average response time is 3 seconds, then it is better to select target interval for small garbage collection about 4-5 seconds. The intention is to make sure that between two small garbage collections some end user requests are completed and thus more memory could be recycled. If many requests are still “in process” then with small garbage collection memory cannot be recycled (still in use) and might be promoted to the tenured generation.

· Ifgc = interval of full garbage collection (in seconds), for example 600 seconds (10 minutes)

The target Ifgc interval should consider the average duration of a user session. If the duration of a user session on average is less than 10 minutes then a target full garbage collection interval of 10 minutes will be ok. If the user session is about 30 minutes long, the target full gc interval should be adjusted to higher value. When full garbage collection is running, but user sessions are still active and the session memory cannot be recycled, the system performance is usually bad with a real danger of crash with “Out of Memory” (OOM) error. Such effect means that we have not performed the memory sizing correctly to reflect the expected load characteristics.

 

Formulas for Java Memory Sizing

 

The total amount of memory which is required to hold all user sessions is calculated like

(F1) Total Sessions Space [MB] =               Number of concurrent users*UserSessionSpace [MB] + Number of new sessions per second*UserSessionSpace [MB] * Ifgc

❗ The user session could have different size after execution of different dialog steps of the scenarios. For “pessimistic” sizing take the maximum measured session size (peak session size) among all user sessions and for “normal” sizing take the average user session size.

❗ Not only the currently active sessions are included in the sizing but also the new sessions which will be created between two sequential collections are considered and planned for session space.

If tenured generation is sized only for the currently alive sessions, then every new created user session may trigger full garbage collection. This full garbage collection will not able to collect enough unused Sessions Space memory, because most of the user sessions are still alive and very soon a new full garbage collection will be required and the frequency of full garbage collections will increase. To avoid this effect in the tenured generation there must be planned capacity to handle new incoming sessions which replace the logged out or expired sessions without need that those unused objects are immediately garbage collected.

 

In some application implementations, classes and methods are created at runtime (so called “runtime class/method generation”). In such cases the size of permanent generation area will depend on the number of concurrent users too – the delta growth per user session will be application specific and determined as additional memory KPI: PermUserSessionSpace [MB/user].

(F1a) Total Permanent Sessions Space [MB] =
Number of concurrent users*PermUserSessionSpace[MB] + Number of new sessions per second*PermUserSessionSpace[MB] * Ifgc

 

There is relation between the number of user sessions and the requests per second, which are processed:

Requests per second = (Number of concurrent users) / (Average response time [s] + Think time [s])

 

Given this, it is easy to calculate the total processing space required by java objects, generated between two sequential small garbage collections in “Isgc interval”

(F2) Total Processing Space [MB] = Requests per second * Processing space [MB] * Isgc

 

❗ With SAP Java Server there is no mechanism to split the total required Framework Space or Perm Framework Space into parts and put it to different JVMs. Therefore the required Framework Space or Perm Framework Space is allocated within the tenured generation and within the permanent generation, of every Java Virtual Machine, i.e. on every Java sever node.

This means that with every Java Server node the tenured generation and permanent generation sizes should always be bigger than framework spaces, i.e.

Tenured generation[MB] > Framework Space[MB]

Permanent generation[MB] > Perm Framework Space[MB]

 

Determine physical memory requirements and Java Server configuration

 

After Framework Space [MB] (also Perm Framework Space [MB]) is measured, and Sessions Space [MB] (also Perm Sessions Space [MB]) and Processing Space [MB] are calculated using formulas F1, F1a and F2, to complete memory sizing remains to

 

Determine the cluster configuration – heap size and number of java server nodes

Heap size [MB] = Framework Space[MB] + (2 * Total Processing Space [MB] + Total Sessions Space [MB]) / (Number of Server nodes)

Perm size [MB] =  Perm Framework Space[MB] + (Total Permanent Sessions Space [MB]) / (Number of Server nodes)

Multiple combinations of Number of Server nodes and Heap (Perm) sizes are possible.

ℹ If more server nodes with smaller heap are preferred, then the total physical memory requirement would be higher due to the fact that each server would have own native code area for initialization of the JVM and that the framework space memory is repeated in each and every server node.

➕ For stability and failover reasons choose java cluster configuration with at least 2-3 java server nodes.

ℹ If less server nodes with bigger heap are preferred, then longer duration of the garbage collection is expected.

➖ Avoid too many server nodes, e.g. more than 10, because of possible increased startup times in cluster.

 

Calculate the total amount of required physical memory

Physical memory [MB] = Number of Server nodes * (Heap size [MB] + Perm size [MB] + Native code area [MB])

 

What to deliver as sizing?

 

In java memory sizing guideline it is not necessary to provide all details of measured memory, neither formulas for calculation.

As result of all sizing calculations, the sizing guideline can look like a table where the sizing expert has decided on the optimal number of server nodes and appropriate heap sizes, for example:

 

Category	Up to … concurrent users	Physical memory requirement [GB] – all java servers	Recommended number of server nodes	Recommended heap size [GB] per server node
S	100	12 GB	2	4
M	1.000	20 GB	2	8
L	5.000	80 GB	6	12
XL	10.000	160 GB	9	16
more	contact SAP	contact SAP	contact SAP	contact SAP

 

The java  memory sizing is application-specific. If more than one application is intensively used in given java environment, the java memory sizing should be calculated for the total usage.

❗ Java Applications, which set in advance memory KPIs for session space (e.g. up to 5 MB per session) and processing space (e.g. up to 10 MB per request) and manage to fulfill it during development will have low memory requirements and good performance.