Skip to Content
Business Trends

Recommended Data Tiering Approaches for SAP and Native Applications

The dilemma at the heart of a series of recent blog posts on data tiering is familiar to many organizations: how to maximize data value, accessibility, and security while minimizing the costs of storing endlessly accumulating information.

Data tiering (read a comprehensive overview here) offers a straightforward, elegant solution to the increasingly vexing problem of data storage: scale and score data so that it can be stored with the solution that offers the best cost to performance ratio. Frequently accessed, high-value data is categorized as “hot” and stored in-memory on the SAP HANA database. Less frequently accessed “warm” data is stored on a lower-cost storage tier managed as a unified part of the SAP HANA database. And, finally, rarely accessed, low-value data is stored on low-cost storage tiers like disk or Hadoop, which remain accessible at any time.

To manage these different temperature categories of data, SAP customers have a number of different tiering options, dependent on the type of application in use. What follows is an overview of those options for SAP HANA native applications; SAP BW on HANA and SAP BW/4 HANA; and Suite on HANA and SAP S/4 HANA.

Hot Data Tiering

As denoted in the diagram above, all the applications share the same in-memory storage capabilities in SAP HANA for mission-critical data that requires real-time access. Non-volatile memory (NVM), also known as SAP HANA persistent memory (PMEM), adds additional  storage capacity for data held in-memory. PMEM offers lower TCO and faster database start-up time after a shutdown than DRAM, but is not a replacement as DRAM is still required for SAP HANA systems.

Warm Data Tiering

When data becomes less mission critical and moves from hot to warm, native SAP HANA, SAP BW on HANA, and SAP BW/4 HANA offer a common warm storage tier option: the Extension Node, with nearly in-memory performance and full SAP HANA functionality (a previous post offers a detailed exploration of the Extension Node). The SAP BW on HANA modelling tool and SAP BW/4 HANA Data Tiering Optimization (DTO) manage the classification and relocation of data between SAP HANA in-memory and Extension Node.

Native SAP HANA applications have an additional warm data storage option, which offers lower costs and similar performance to the in-memory Extension Node, called Dynamic Tiering. Dynamic Tiering is a disk-store option that is part of the SAP HANA landscape and is accordingly easy to implement and manage but does not support certain SAP HANA capabilities, such as geospatial and time series functions. (Take a closer look at Dynamic Tiering for warm data management).

For native SAP HANA applications, the Data Lifecycle Management (DLM) tool, which is part of the SAP HANA Data Warehousing Foundation (DWF), facilitates warm data tiering by enabling rule-based movement of data between SAP HANA in-memory and Dynamic Tiering or Extension Node.

Suite on HANA and SAP S/4 HANA implement Data Aging for warm data by enabling Paged Attributes within SAP HANA. Paged Attributes are only applicable in the context of Data Aging. For Suite on HANA, Data Aging requires the implementation of aging objects in Suite Applications. Only the documented aging objects listed in the online help section for Suite on HANA or SAP S/4HANA can be moved between SAP HANA in-memory and disk. Periodic aging runs are executed within the Data Aging framework that take into account the data of aging objects paged on disk. During the aging run, all tables of an aging object are partitioned, in a part, with current “hot” data and historical “warm” data. All historical partitions of a table are marked as page-loadable, which means the columns of historical partitions are defined as Paged Attributes.

Paged Attributes offers a targeted warm tier solution, allowing users to load and unload table columns page by page from the SAP HANA persistence layer into memory as needed, avoiding the added burden of loading entire columns of tables at once. Additionally, and even more importantly, Paged Attributes receive a high unload priority from the SAP HANA database: they are paged on disk first when SAP HANA decides to free up memory. (Find out more about Paged Attributes.)

Cold Data Tiering Options

Just as all hot and warm data tier storage is housed on the SAP HANA database, all cold data tier storage exists on external stores, such as SAP IQ, SAP HANA Vora or Hadoop. As we already know from warm data tiering, applications have different SLA requirements and therefore offer their own cold storage tiering approach. And the good news? Cold data tiering tools are mostly the same as what’s used for warm data tiering. (Find an in-depth look at cold data tiering here.)

Native SAP HANA applications use the DLM tool to support Hadoop or cloud storage destinations. DLM facilitates data tiering by enabling a rule-based and seamless movement of data from hot or warm storage to the appropriate cold data store, and vice versa. (Check out this DLM overview or examine DLM in technical detail.)

SAP BW on HANA and SAP BW/4 HANA manage the classification and relocation of data for read only and sporadic reporting access with Data Tiering Optimization (DTO) or Near-Line Storage (NLS) for external storage in SAP IQ. If using Hadoop or similar systems for external storage, DTO—in conjunction with SAP Data Hub, SAP HANA Vora or the SAP HANA Spark Controller—is used to manage cold data. (Learn more about using DTO for cold data tiering)

Finally, Suite on HANA and SAP S/4 HANA users can archive cold data on external stores with the ILM Store, an ABAP solution that creates archive files using SAP NetWeaver as the application server. In order to read back these archive files, for auditing purposes for example, the files must be converted and imported with ILM. Regardless of this extra step, the ILM Store reduces the data footprint in SAP HANA by allowing for cold data storage on the less costly SAP IQ database system and external solutions like Hadoop. ILM specifically provides a Hadoop Connector to store archive data files to HDFS (Hadoop Distributed File System). The connection parameters are maintained in the ILM Store to establish the communication with the Hadoop instance. (Find more on ILM)

Comprehensive Data Tiering Options

The comprehensive approaches to data tiering detailed within this post and throughout our Data Tiering blog series provide a number of data storage and access options, each designed to provide SAP HANA customers with maximum data value at the minimum cost. We’d be happy to answer any questions you may have about any of the outlined SAP Data Tiering recommendations—simply contact us directly via email or use the comments section below and we’ll get back to you immediately.

/
8 Comments
You must be Logged on to comment or reply to a post.
  • Just noting the comment about DT not supporting “Series” data, I assumed from reading the administrators guide that Multistore Tables don’t support Series, but Extended stores do?

    It would be good to know if this is the case in practice.

  • Dynamic Tiering (DT), no matter how you get there, does not support the HANA time series data types. Supporting those types is a function of the engine, not how HANA interfaces with the engine. Both multistore and extended tables all point to DT, so neither would support time series data. Or Graph. Or Geospatial. These are features that must be implemented within DT for support.

    The list of supported types can be found in the “SAP HANA Dynamic Tiering: Administration Guide” –> System Administration –> Supported Data Types.

    For HANA 2 SP03, this is the link:
    https://help.sap.com/viewer/269740c67eca42a3b4ffbd376b406fbe/2.0.03/en-US/3d03a015daee45879e0450ebff6f92ed.html

    If these types and processing are critical, we have other stores to meet those needs. For instance, you could use Extension Nodes, Paged Attributes, or the upcoming Native Storage Engine (expected in SP04).

    Mark

  • Hi,
    I have 3 questions regarding de Cold storege on hadoop:

    1. Is possible to have hot storage and cold storage, without having the warm storage level?

    2. For hadoop cold storage, in some articles seems that the DTO option is not available, and only it is possible use NLS options. Look:
    “External Tier (cold): The data is stored externally (SAP IQ, Hadoop or SAP Vora). Please note that Hadoop-based external storage is currently only possible via SAP NLS data archiving processes and not via DTO.”
    Is it and old situation and now it’s posible to use DTO for hadoop cold storage?

    3. To use hadoop as cold storage, is needed some SAP DATA HUB component? This comment in you article makes me confused about it: “If using Hadoop or similar systems for external storage, DTO—in conjunction with SAP Data Hub, SAP HANA Vora or the SAP HANA Spark Controller—is used to manage cold data”

    Thanks a lot for you answers.
    Laura

  • Hi Ruediger,
    for S/4 HANA users cold storage defines as archiving storage in SAP IQ by ILM store, unlike native HANA application access to cold storage is possible through SAP Data Hub and the SAP HANA Spark Controller, my question is: can S/4hana application access to cold storage in the same way as native HANA application, for example, SQL queries and etc.?
    Thanks

  • Hi,

    thank you for your post, would you please give me some reference as literature (seminar paper or published articles) which can I reference them in my research.

    Thanks

  • Hi,

    I do have a customer who is interested in NLS. We discussed this and two Questions came up:

    • My Customer does not have a separate BW-System but is using BW-Embedded on SoH 7.5 Netweaver which is working fine. Is NLS also supported for the BW-Embedded Scenario then?
    • Database: NLS seems to require IQ-DB as NLS-Database. As my customer uses internally maxDB on various Systems the Question came up if the database of the NLS-System can
      also be maxDB instead of IQ-DB. Admins would prefere  maxDB as they are not familiar
      with the IQ-DB at all.
    • In case IQ-DB is the only NLS-Database supported, how about the Costs of that database?
      Is it for free when we use it for BW-Embedded NLS and if not who to calculate the Costs then?

    Really would appreciate your help on this. Thank you.

    Kind regards,
    Klaus