Hello SYB DBAs,

you may be noticed a message in your DB log which looks like this:

“Could not allocate memory using Huge Pages. Allocated using regular pages. For better performance, reboot the server after configuring enough Huge Pages.”

Don’t worry about it, if no Huge Pages are available, the system automatically switch back to the normal page size of the OS. For example in Linux the regular memory page size is 4KB.

The huge page size you can find out via command:


cat /proc/meminfo | grep Hugepagesize
Hugepagesize:       2048 kB



Hugepages speed up the memory management.

Hugepages allocated in one piece and can’t be paged out into the paging space.

So you can ensure that the DB memory never be paged out.

Please read also this notes about HugePages and Sybase/SAP ASE:

2021541 – SAP ASE runs slow due to table scans with regular page allocation

1805750 – SYB: Usage of huge pages on Linux Systems with Sybase ASE

For example if the sybase instance need 10GB without Hugepages:


10*1024*1024/4kb=2621440  (4kb Pages)
Every Page needs a 8byte entry in the page table 2621440 * 8byte = 20971520 => 20MB

For example if the sybase instance need 10GB with Hugepages:


10*1024*1024/2048kb=5120 (2mb Pages)
Every Page needs a 8byte entry in the page table 5120 * 8byte = 40960 => 40KB



This values need to be multiplicated with the user processes, but the shared memory must be considered!

If you need more information about the Page Table, TLB and Huge Pages you should have a look at Stefan Koehlers blog which is regarding oracle but however the facts are the same:

http://scn.sap.com/community/oracle/blog/2013/06/17/oracle-myths-and-common-misconceptions-about-transparent-huge-pages-for-oracle-databases-on-linux-uncovered

If you want to use HugePages follow this steps as root

1) calculate the needed memory


select cast(sc.name as varchar(30)) as name, (scc.value*2/1024) as MB
from sysconfigures sc, syscurconfigs scc
where sc.name like 'total%memory' and sc.config = scc.config
go
name                           MB
------------------------------ -----------
total logical memory                  6028
total physical memory                 6400



=> Round the ASE “max memory” (total physical memory) setting up to the next closest 256MB, divide by the Hugepagesize, and configure at least that many huge pages. “

=> 6400 / 256mb = 25 => OK

2) calculate the needed HugePages

When ASE uses huge pages, it will allocate memory to the nearest multiple of 256Mb so don’t configure the ASE exactly to the maximum number of huge pages, but leave a small amount unused (e.g. 300/400 Mb). In case a small increase in size is needed due to the change of a Sybase config you will not get into trouble.


6400+400=6800
6800*1024/2048=3400
vi /etc/sysctl.conf
vm.nr_hugepages=3400



3) activate the kernel settings, if enough memory is available

sysctl -p

if the memory is not free, restart the server or add memory

4) Allow Sybase ASE owner to make use of available HugePages


vi /etc/security/limits.conf
<Sybase ASE OS owner>   soft memlock unlimited
<Sybase ASE OS owner>   hard memlock unlimited


5) check the config


cat /proc/meminfo | grep Huge
HugePages_Total:    3400
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB



6) start the Sybase/SAP ASE and you will see this message in the logs

“Allocated memory using Huge pages.”

7) after DB start


cat /proc/meminfo | grep Huge
HugePages_Total:    3250
HugePages_Free:     1655
HugePages_Rsvd:     1605
HugePages_Surp:        0
Hugepagesize:       2048 kB


If you get the same error as before extend the HugePages.

more offical info:

1805750 – SYB: Usage of huge pages on Linux Systems with Sybase ASE

Before and after process details

Here are the before and after effects regarding the dataserver (sybase DB process under unix) after a fresh restart:

Before:


pmap <sid>
START               SIZE     RSS     PSS   DIRTY    SWAP PERM MAPPING
0000000000400000  32736K  15460K  15460K      0K      0K r-xp /sybase/SMP/ASE-16_0/bin/dataserver
00000000025f7000   5692K   2664K   2664K    216K      0K rwxp /sybase/SMP/ASE-16_0/bin/dataserver
0000000002b86000   2488K   1196K   1196K   1196K      0K rwxp [heap]
0000000142df4000 6291456K 1507196K 1507196K 1507172K      0K rwxs /SYSVba156435
00007fffed33e000     20K     16K      2K      0K      0K r-xp /lib64/libnss_dns-2.11.3.so
[...]
Total:           6507456K 1530568K 1528772K 1509564K      0K


ps auxw | head -1; ps auxw | grep <sid>
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
sybsmp   31464 15.5  9.2 6507456 1516852 ?     Ssl  08:56   0:24 /sybase/SMP/ASE-16_0/bin/dataserver -sSMP -d/sybase/SMP/sybsystem/master.dat -e/sybase/SMP/ASE-16_0/install/SMP.log -c/sybase/SMP/ASE-16_0/SMP.cfg -M/sybase/SMP/ASE-16_0 -i/sybase/SMP -N/sybase/SMP/ASE-16_0/sysam/SMP.properties

=> RSS = 1,5GB

=> VSZ = 6,5GB

After:


pmap <sid>
START               SIZE     RSS     PSS   DIRTY    SWAP PERM MAPPING
0000000000400000  32736K  16096K  16096K      0K      0K r-xp /sybase/SMP/ASE-16_0/bin/dataserver
00000000025f7000   5692K   2664K   2664K    216K      0K rwxp /sybase/SMP/ASE-16_0/bin/dataserver
0000000002b86000   2648K   1244K   1244K   1244K      0K rwxp [heap]
00002aaaaac00000 6291456K      0K      0K      0K      0K rwxs /SYSVba156435
00007fffed33e000     20K     16K      0K      0K      0K r-xp /lib64/libnss_dns-2.11.3.so
00007fffed343000   2044K      0K      0K      0K      0K ---p /lib64/libnss_dns-2.11.3.so
00007fffed542000      4K      4K      4K      4K      0K r-xp /lib64/libnss_dns-2.11.3.so
00007fffed543000      4K      4K      4K      4K      0K rwxp /lib64/libnss_dns-2.11.3.so
00007fffed544000     48K     20K      0K      0K      0K r-xp /lib64/libnss_files-2.11.3.so
00007fffed550000   2044K      0K      0K      0K      0K ---p /lib64/libnss_files-2.11.3.so
[...]
Total:           6507616K  24056K  22094K   2440K      0K



vmsaplnx02:~ # ps auxw | head -1 ; ps auxw | grep <sid>
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
sybsmp    1716  9.6  0.1 281536 22712 ?        Ssl  09:12   0:20 /sybase/SMP/ASE-16_0/bin/dataserver -sSMP -d/sybase/SMP/sybsystem/master.dat -e/sybase/SMP/ASE-16_0/install/SMP.log -c/sybase/SMP/ASE-16_0/SMP.cfg -M/sybase/SMP/ASE-16_0 -i/sybase/SMP -N/sybase/SMP/ASE-16_0/sysam/SMP.properties

=> RSS = 23MB

=> VSZ = 282MB

If you compare the RSS/VSZ of the 2 ouput you will see the benefit.

Hope this howto helps you to use your memory ressources optimal.

Thanks for reading and sharing!

Please feel free to ask or get in contact directly if you need if you need assistance.

Best Regards,

Jens Gleichmann

Technology Consultant at Q-Partners Consulting und Management GmbH (www.qpcm.eu)

References

Virtual memory – Wikipedia, the free encyclopedia

Resident set size – Wikipedia, the free encyclopedia

[Oracle] Myths and common misconceptions about (transparent) huge pages for Oracle databases (on Linux) uncovered

2021541 – SAP ASE runs slow due to table scans with regular page allocation

1805750 – SYB: Usage of huge pages on Linux Systems with Sybase ASE

To report this post you need to login first.

3 Comments

You must be Logged on to comment or reply to a post.

  1. Mark A Parsons

    Would be interested in comments about the need to set hugetlb_shm_group.

    Last year I setup a new ASE on linux and in addition to the steps listed above, I had to add the following to sysctl to ensure the ‘sybase’ group could access hugepages:

    NOTE: Assume the ‘sybase’ group id is 504 (per /etc/group)

    =============== /etc/sysctl.conf

    … snip …

    vm.hugetlb_shm_group = 502

    … snip …

    ===============

    (0) 
    1. Jens Gleichmann Post author

      Hi Mark,

      thanks for reading and the input.

      You have to use hugetlb_shm_group if you reserve the huge pages exclusively for this group. So may be another process has allocated some huge pages before the sybase ASE could access them.

      Regards,

      Jens

      (0) 
  2. Rao Bheemarasetty

    Apart from the above settings, one more import OS level setting need to be considered with hugepages is kernel.shmall. If this value not configured properly the ASE will not start.

    If the ASE is configured to use X number huge pages please set this(kernel.shmall) config. to fit in the configured huge pages.

    (0) 

Leave a Reply