Let Us Be Persistent – HANA Adoption of Non Volatile Memory
I have been keeping track of Non-Volatile memory for HANA from the time I have read below article and watched several videos on non-volatile memory technology.
Finally, on 2nd April 2019 Intel has announced the general availability of Optane DC persistent memory. Optane is a new tier of memory that’s faster than SSD storage and less expensive than DRAM. But unlike DRAM, Optane is persistent i.e. it will keep the information stored on it even after system reboot. For thorough insight on Optane Technology, you can refer below whitepaper.
Technology Brief on Intel Optane Technology: Link
Intel has also introduced its 2nd Generation Xeon Scalable Processor, also known as Cascade Lake. The interesting feature of this new CPU is the support of new tier of memory (Optane DC Persistent Memory). The highest configuration available in the market that will support memory will 28 Cores/CPU.
When HANA got released, the initial core to memory ratio was 756 GB/socket for OLTP and 256 GB/socket for OLAP which got changed to 1.5 TB/socket in 2017-18 (Link). Now with the introduction of Intel’s new Memory, SAP HANA becomes the first major database platform optimized for the full potential of Intel Optane DC persistent memory which can support core to memory ratio of 4.5 TB/socket which is 6x time more than initial core to memory ratio.
Now the infrastructure costs for SAP HANA servers will be reduced by combing the deployment of Optane DC persistent memory and DRAM alongside each other in memory channel. Just to be clear HANA database cannot solely run on Optane DC persistent memory, we still require DRAM for operations.
Persistent memory can be used for the main storage of column store table that is typically the dominating factor of data space consumption in SAP HANA environments. Other areas like delta storage, caches, intermediate result sets or row store remain solely in dynamic RAM (DRAM). Note that if persistent memory storage is enabled, data is still written to the data volumes but is simply ignored.
As column store main fragment will reside in persistent memory, so when our HANA system get restarted it won’t unload and load data during restart operation. It has been said in Intel Data Centric Innovation that the restart of 6 TB HANA server will get reduced from 50 minutes to 4 minutes. Really a thing every customer was looking for as this will save considerable time for any maintenance activity or uninterrupted outage.
Just for the information, data in persistent memory isn’t redundant to data in DRAM. Column store main storages stored in persistent memory aren’t stored in DRAM. So, the overall amount of required memory doesn’t change when using persistent memory.
How core to memory ratio of 4.5 TB/socket has been calculated/decided for HANA? (This is basically my understanding)
On 2nd April 2019 Intel has also introduced 2nd Generation Intel Xeon Scalable Processors with 28 Cores/CPU which can support Intel Optane Memory and has 6 memory channel. Each memory channel can handle 2 memory slots (this are the slot where your insert DIMM which can be PMEM or DRAM). So, in total Intel Xeon Scalable Processor have 12 DIMM (6 x 2) slots. Minimum CPU available for any server is 2 (Link).
2 CPU x 6 Memory Channel x 2 DIMM = 24 Memory Slot/Server
As per SAP, to achieve highest memory performance, all DIMM slots must be used in pairs of DRAM DIMMs and persistent memory DIMMs, i.e. the system must be equipped with one DRAM DIMM and one PMEM DIMM in each memory channel. Earlier we needed same set of DRAM DIMM module on the server i.e. all memory slot filled up with any one of the 32/64/128 GB DIMM.
So, in these 24 memory slots we will have 12 slots filled up with PMEM and 12 slots with DRAM
PMEM: 12 x 512 GB = 6 TB
DRAM: 12 x 128 GB = 1.5 TB
Total memory: 7.5 TB
The only concern here is the availability of NVRAM, as there will be huge demand for persistent memory by hardware vendors and cloud giants. The reason for this concern is, at times there is/was shortage of 128 GB DRAM DIMMs in the market as well.
Persistent memory is supported with SAP HANA 2.00.035 and higher, but we need appropriate underlying hardware in place to take the advantage. There are vendors which are already in process of getting HANA appliance certified on new Xeon Scalable Processor with Persistent memory by SAP. The operating systems supported for Persistent memory is SLES 12 SP4 or SLES 15 or RHEL 7.6. SUSE Linux Adds Support for Intel Persistent Memory on SAP HANA (Link)
So even from cloud vendors, we can expect high memory VM instance instead of bare metal. Google Cloud is the first public cloud provider to offer virtual machines using Intel Optane DC Persistent Memory in Q3’18 as part of alpha program with 7 TB virtual machine. Recently in #GoogleNext19, Google has announced 12 TB virtual machine (Announcement Link). They will be providing even large Optane-based VM
More info about Google Cloud – Intel Optane DC Persistent Memory
Mixed combinations with primary systems having persistent memory and secondary systems without persistent memory and vice-versa are supported. System replication can be used to move a system from servers without persistent memory to servers with persistent Memory and vice versa. This is helpful if you want to set up SAP HANA System Replication Cost Optimized Scenario. One of the scenario I can think of is below, but there can be many combination of deployment option possible.
Node 1 – Production: 3 TB (2 Socket – All memory slot filled up with DRAM)
Node 2 – Production HA + Non-Production: 7.5 TB (2 Socket – Mixed combination)
Currently there is no benchmark result released comparing HANA running on DRAM vs HANA running on mixed combinations, but we can expect a slight degradation in performance of HANA when it runs on mixed combinations. You can watch the video on Intel Optane DC Persistent Memory Performance Overview to get some idea on the performance benchmark of Persistent memory. Eagerly waiting for this comparison from sap, hardware vendors or partners.
From the configuration point of view, SAP HANA only supports DAX enabled (Direct Access) file systems. Each DAX file system enables the access to persistent memory connected to one CPU socket. The default behaviour for using persistent memory is determined automatically: the system tests for the DAX filesystem at the defined persistent memory basepath (/hana/pmem/<sid>). If the hardware is installed and the basepath correctly configured pointing to the persistent memory mount, then all tables will use persistent memory by default.
It will be interesting to see this new file system configuration/layout. Looking forward to find out how HANA perform with PMEM
To join the Live Virtual Expert series to learn about the new capabilities in the upcoming release (HANA 2.0 SPS 04) be sure to register via the following links:
Also, you can refer below links to read more on Persistent Memory
Feel free to drop a comment on your thought for Persistent memory for HANA and what you expect from new HANA 2.0 SPS 04.