Skip to Content
Business Trends

SAP HANA Native Storage Extension: A Native Warm Data Tiering Solution

With the exponential growth of data showing no signs of slowing, customers looking to lower their total cost of ownership (TCO), while still leveraging their vast reservoirs of data to maximum effect, must continuously manage their data storage costs without forfeiting the high performance they’ve come to expect from SAP HANA. A few data tiering options with different price/performance characteristics are already available, but soon SAP customers will have another option in their data tiering arsenal: a native solution for warm data tiering—the SAP HANA Native Storage Extension.

A Quick Data Tiering Refresher

A quick data-tiering refresher—mission-critical hot data is retained in-memory on the SAP HANA database for real-time processing and analysis. Less frequently used warm data is stored in a lower cost tier, but still managed as a unified part of the SAP HANA database. Rarely used, voluminous cold data is located on the lowest cost storage. Regardless of location, all data remains accessible at any time.

The new SAP HANA Native Storage Extension (NSE) adds a native warm data tier to the SAP HANA database. Any customer-built or SAP-built HANA application that is challenged by ballooning data volumes can leverage this deeply integrated warm data tier. NSE increases SAP HANA data capacity at a low TCO through a simple, scalable landscape that offers great performance. Supporting full SAP HANA functionality and all SAP HANA data types and data models, NSE complements—without replacing—the Extension Node and Dynamic Tiering warm data options. NSE is supported for both on-premise SAP HANA systems, and SAP Cloud Platform, SAP HANA Service.

How Does SAP HANA Native Storage Extension Work?

While hot data is ‘column loadable’, residing completely in-memory for fast processing and loaded from disk into SAP HANA memory in columns, the SAP HANA Native Storage Extension allows a user to specify certain warm data as ‘page loadable’, which is then loaded into memory page by page as required for query processing. Page loadable data does not need to reside completely in-memory, like column loadable data.

NSE reduces the memory footprint of the SAP HANA database with expanded disk capacity and an intelligent buffer cache that transfers pages of data between memory and disk. Query performance differences may be noticeable between warm data and hot data.

NSE Warm Data Tiering

Additional Database Capacity with SAP HANA Native Storage Extension

SAP HANA scale up systems are supported with this upcoming initial NSE release and scale out support will follow in a later release. With NSE, you will be able to expand SAP HANA database capacity with warm data on disk up to about 4x the size of hot data in memory. A relatively small amount of SAP HANA memory for the NSE buffer cache will be needed for paging operations, as the buffer cache can handle 8x its size of warm data on disk. As an example, a 2TB SAP HANA system without NSE equates to a 1TB database in memory. With NSE and the addition of a 500GB buffer cache, you can expand your 1TB database to a 5TB database: 1TB of hot data, 4TB of warm data, and a 500GB buffer cache to page data between memory and disk.

Next Steps

We are excited to roll out this native SAP HANA warm data tiering solution that boasts full SAP HANA functionality at a lower TCO for customers. Want to learn more about SAP HANA Native Storage Extensions? Find out more here or drop me a question in the comments section below.

21 Comments
You must be Logged on to comment or reply to a post.
  • Hi Kumar,

    NSE is an extension and enhancement of the Paged Attributes capability which S/4 already uses for data aging. In that context S/4 is using a version of NSE today, with room to take more advantage of the enhanced capabilities in the future.

    BW is currently focused on the use of Extension Nodes for warm data tiering. BW may evaluate NSE for future use.

    Thanks
    Rob

  • Hi Robert,
    In NSE solution, should we consider the presidency layer(Disk, Data Volume) of HANA as a warm tier or we need to deploy extended storage (Disk) e.g. IQ Sybase which tightly integrated with HANA DB as warm tier.
    Also a possibility of adoption persistent memory(PMEM) as a solution for hot tier and NSE for warm tier same time.
    Thanks,
    Arash

  • Hi Arash,

    NSE is warm data tiering option within the HANA system. Like DT, NSE pages or moves data back and forth from disk to a cache when needed to process queries. A big difference between NSE and DT is that NSE is a feature of the HANA index server and runs as part of that process. Data that is assigned to be “page loadable”, meaning that it is being managed by NSE, is written to the same files on disk as in-memory “column loadable” HANA data is written to. For in-memory column loadable HANA data, HANA only needs to read the data from disk immediately after start up. In contrast NSE page loadable data can be read into the buffer cache as needed, the flushed or “ejected” from the cache either when no longer needed or when HANA needs to free up space in the buffer cache for other pages. If the same page is required again then it will be read from disk again.

    Since NSE shares the same disk storage files or “disk persistency” as in-memory HANA data, there is no need to also configure DT. Our current recommendation is that new data tiering projects should evaluate NSE as the first warm data tiering option. Key limitations for NSE in the HANA 2.0 SPS 04 release are that it supports scale-up (single node) systems only and supports up to 10TB of warm data. If you require warm data tiering for scale out (2 or more node) systems and/or data volumes in the range of 11TB -100TB then DT should be considered.

    Yes you can use PMEM storage for in-memory data and still have other tables, partitions, or columns assigned to page loadable storage using NSE.

  • HI Robert,
    Thanks for detailed information,
    I have few more questions,Is NSE compatible for BW on HANA environments.also help me with NSE configuaration guides and administrations Guides.

    Thanks and Regards,
    Kiran

  • Hi, Robert!
    We would like to actively use NSE after our planed upgrade project to SPS 04. In the documentation I can not find any more specific steps that need to be executed before using NSE. Is this true? Isn’t there some kind of configuration necessary before using it

    Thanks in advance,
    Markus

  • Hi PRJSYN,

    It is up to individual SAP application teams to decide which HANA features to use. In the case of BW, the BW team is considering adding support for NSE in 2020.

    For NSE documentation I would recommend starting from the SPS 04 New Features section of the documentation which then links to more detailed NSE documentation: https://help.sap.com/viewer/42668af650f84f9384a3337bcd373692/2.0.04/en-US/c71469e026c94cb59003b20ef3e93f03.html

    We will also be delivering a hands-on session for NSE at TechEd this year. The session title is DAT370 – Operating an SAP HANA System with SAP Native Storage Extension

    Thanks
    Rob

  • Hi Markus,

    Data can be allocated to NSE storage at a table, column, or partition level. For existing objects this is done using an ALTER TABLE command.

    There is a recording available for the “What’s New in HANA 2.0 SPS04: HANA Data Tiering Options” presentation here: https://event.on24.com/eventRegistration/EventLobbyServlet?target=reg20.jsp&partnerref=hanablog&eventid=1945137&sessionid=1&key=AE112CD83A14398AFBBAE01766DD6EF8&regTag=465648&sourcepage=register

    You can also work through the NSE documentation section in the SAP HANA Platform documentation: https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.04/en-US/4efaa94f8057425c8c7021da6fc2ddf5.html

    This includes a section specifically covering “Getting Started With the Native Storage Extension”: https://help.sap.com/viewer/6b94445c94ae495c83a19646e7c3fd56/2.0.04/en-US/7ab2658cce56438c92f1cc7b13c50597.html

    Thanks
    Rob

  • Hi Robert,

    I have read through your blog and also few more blogs over NSE. I feel NSE is a good option among the various data aging solutions as it is easier to manage without any additional nodes. I have some specific queries regarding the same, it would be great if you can help with it:
    1. In my workplace we have SAP BW on HANA 7.5 SP5 with HANA 1.0 SP12. I look NSE here to unload some data to the warm storage instead of having all the data in memory. In this regard, when likely the NSE be made available for SAP BW on HANA?, what will be the supported versions?, what is the plan of SAP for NSE for SAP BW on HANA?
    2. If NSE is made available for SAP BW on HANA, will it be possible to enable NSE only to new providers like ADSO or will also there be an option to set aside some data from older providers like HANA optimised DSO (Standard, Write optimized, Direct update), Cubes etc. to the NSE layers.
    Looking forwards for to hearing from you.

    Thanks and Regards,
    Adi

  • Hi Adi,

    The BW team is targeting support for NSE in Q1/2020. I’m not sure what BW version that will be. You may want to submit a question through the SAP Community Network (https://answers.sap.com/questions/ask.html) and tag it “BW SAP HANA Data Warehousing”. That would also be a good forum to ask about what BW objects will be supported for data tiering.

    Thanks
    Rob

    • Hi Udo,

       

      XSA is an application server layer that can run as part of a HANA system. Applications built to run on XSA still connect to a HANA database and access HANA tables. The use of NSE is configured at the table level as part of the CREATE TABLE or ALTER TABLE statements and is transparent to an application that is using the table. Note that the use of NSE – or assignment of a PAGE LOADABLE load unit – can also be controlled when using HDI for the physical data modelling. With HDI, the PAGE LOADABLE load unit assignment is including the hdbtable table definition.

       

      The end result is that there is no XSA specific syntax required to use a table that is either fully or partially assigned to be PAGE LOADABLE (using NSE) whether that table is being accessed through SQL statements or through calculation views.

       

      Thanks

      Rob

    • Hi Sandeep,

       

      NSE is a feature of the core HANA indexserver process and is included and enabled in all HANA 2.0 SPS 04 installations. No additional license is required to use NSE.

       

      Thanks

      Rob

      • Hello Robert,

         

        does it mean, that NSE is also available for customers of HANA with runtime license for BW?

        Thank you in advance for an information, I cannot find it anywhere.

         

        Best regards:

        Lukasz

        • Hi Lukasz,

           

          Yes, as I said NSE is a feature of the core HANA indexserver and is part of all HANA 2.0 SPS 04 systems. That includes runtime systems. While there is no separate license required for NSE, the buffer cache used to manage page loadable data with NSE is part of licensed HANA memory capacity.

           

          BW4/HANA added support for NSE with their SP 04 release in March.

          Thanks

          Rob

  • Hi Rob,

    In your example, enabling NSE means we need a 500GB buffer cache. I assume this will be carved out of the 2TB and hence usable capacity will be 1.5TB which means database size (hot) will be 750GB? Is this a correct understanding while we look at sizing?

    And, even though I get the idea of data temperatures, it looks like we are going back to the old school of buffering data from disk and hence HANA becomes like a hybrid database with in-memory and traditional database styles. I assume the buffer cache will use LRU mechanisms to page out and there will be an impact on performance of queries accessing data stored in NSE. Is it just a price to performance call?

    • Hi Sivaramakrishnan,

       

      In the example given the total HANA memory capacity is being increased from 2 TB to 2.5 TB with the addition of 500 GB of HANA memory capacity for use by the buffer cache. If you were to carve the buffer cache out of the existing 2 TB of HANA memory capacity then following the default sizing guidance you would allocate a maximum of 400 GB of memory to the buffer cache and your maximum data volume would be 800 GB of hot in-memory data + 3.2 TB of warm page loadable data for a total database capacity of 4 TB.

       

      Yes NSE implements database paging for HANA and yes the basic caching algorithm is an LRU algorithm. Since page loadable data using NSE is not always in memory, then operations on that warm data are expected to take longer than operations on pure in-memory data.

       

      Generally speaking, customers who are implementing NSE are looking to manage TCO by reducing their HANA memory footprint while scaling overall HANA data capacity and maintaining reasonable performance for older less frequently accessed data.