Business Trends
SAP HANA Dynamic Tiering
While RAM prices have significantly dropped (from $6 Million per GB in 1980 to $5.5 per GB in 2013) in last few years, data has also been significantly growing (data doubles every 18 months). To meet today’s business requirements, it is vital to get instantaneous responses on current data. At the same time, it is also important to have the ability to transparently access large amounts of data when needed.
The dynamic tiering option in SAP HANA SPS09 gives the ability to keep the data in either memory or on the disk in a columnar format. Data is not duplicated. Dynamic tiering option helps users to choose memory for hot data and disk for warm data, helping to strike the right price/performance balance. To do so, you define the table as “extended table” using an SQL CREATE statement. These tables are like any other SAP HANA tables except for the fact that they are created on the disk and not in memory. From the application developer point of view, these tables can be queried and modified using standard SQL statements, like any other SAP HANA tables. You can join extended tables with in-memory tables and at any point of time, you can convert extended tables into in-memory tables and vice versa, with an SQL ALTER statement.
Dynamic tiering also works with multitenant database containers and you can configure extended storage for each tenant.
When I showed the dynamic tiering system demo at TechEd&&-code Las Vegas I got some interesting questions from people stopping at the SAP booth. Here are a few of them:
1) Is it possible to move data from in-memory table to the extended table?
Yes. You can write a procedure that moves data from in-memory table to the extended table, based on some conditions. Using the job scheduler in SAP HANA XS engine, you can then periodically run the procedure and physically move the data from the in-memory table to the extended table. You can learn more about SAP HANA XS Engine job scheduler from SAP HANA Help tutorial and video from Thomas Jung.
2) Since all in-memory tables are stored also in disk for durability (to restart when power goes off), then, what is the difference between in-memory tables and extended tables?
With extended tables, data is expected to be in the disk and algorithms used to access the data are optimized for disk-based data. With in-memory table, data is expected to be in memory and algorithms used to access data are optimized for in-memory data. When data in extended tables is updated or new data is inserted into them, the changes are made in the disk. With in-memory table, data is updated in the memory. Of course, logs are written to the disk to ensure durability in both cases.
That being said, in some cases, it is possible that data in extended tables is cached in memory and in-memory tables are not loaded in memory.
3) Does dynamic tiering supports backup and recovery?
Yes. SAP HANA manages backup for extended tables. System replication for extended tables is planned for next release.
4) How is Near-Line Storage (NLS) in SAP Business Warehouse (BW) option different from dynamic tiering?
NLS in SAP BW uses SAP IQ as a cold, read-only, historical archive. Dynamic tiering is used to store warm data. With dynamic tiering, SAP HANA gives the ability to create extended tables on the disk. These extended tables are local to SAP HANA and queried or modified like any other HANA table. You can use both dynamic tiering and NLS at the same time for SAP BW on HANA. In this case, SAP BW will handle data management for you. SAP is working on making dynamic tiering more suitable for additional use cases.
In summary, dynamic tiering is a new SAP HANA feature introduced in SPS09 that gives the ability to store frequently used data (hot) in memory and less frequently used data (warm) in the disk.
Following are some of the additional resources to learn more about dynamic tiering feature:
- Dynamic Tiering @ SAP Help portal
- TechEd&&d-code session on dynamic tiering
- What is new in SPS09 – Dynamic Tiering? – slides
- SAP HANA Academy Video – SAP HANA Dynamic Tiering: Introduction [SPS09]
Please let me know if you have any additional questions.
This is a capability some of the customers have been asking for. Thanks for supporting and documenting it.
Why the changelogs arent made extended tables to be moved to dynamic tiering ?
Yogesh, It is not clear to me what you mean by "changelogs arent made extended tables" because we have change logs in SAP Business Suite and logs (redo logs) in SAP HANA. Could you please clarify?
Very well explained
The initial concept of HANA as in-memory database now doesn't make sense .
There is no change in the initial concept of HANA as an in-memory database. With Dynamic Tiering, the user gets the option to keep less frequently used data (warm data) on the disk rather than keeping it in memory.
SAP HANA is still an in-memory AICD compliant columnar database that can handle both transactions and analytics on a single copy of data in memory.
Hi Sai, very well explained and a good read. Are there are additional licensing implications to set up dynamic tiering?
For combined reporting on hot and warm stores, I understand you can create simple DB views or information views Union -ing or joining the two tables - but filters as where conditions on these views - will they be pushed down to the table level? Or would it first Union all data from both tables first before applying any filters on them?
Thanks,
Anooj
Hello Sai,
Just want to check with you can SAP HANA Dynamic Tiering feature of SPS9 can be useful in SoH Migration or S/4 HANA migration. Can we have hot data or warm data implement in SAP ECC system running on HANA by using extended tables?
As per me SAP HANA Dynamic Tiering feature is more useful for SAP BW (NLS approach)or other project for Big Data or any other native application (data provision based) where data can become candidate for warm or hot data.
Thanks,
Bhupesh
Dynamic Tiering (DT) as it stands today, requires implementation work within the application that makes use of the capability. At this time, neither Business Suite nor S/4 Hana have done the work to implement dynamic tiering. As dynamic tiering evolves to be more native to the HANA database, these applications will likely pick it up for large data volume use cases. The BW team has undergone the effort, however, and DT is nicely integrated with BW.
Hi,
I would like to make test drive in aws for SAP HANA Dynamic Tiering. Can anyone help me for deploying DT in AWS?
Regards,
Siva.
The HANA Developer Edition can be hosted on AWS and the latest version for AWS includes Dynamic Tiering. Instructions on setting up an AWS instance of the HANA Developer Edition are available here: http://go.sap.com/developer/tutorials/setup-hana-for-cloud.html
Thanks for the information on SAP Dynamic Tiering.
Does this mean that Data/tables would be duplicated as in "Extended Tables" and in "Disc like SSD storage" as per durability concept? This may contradict with SAPs initial statement as "Store data once and Access anywhere"!
Data is not duplicated. If table is defined as an in-memory table, then it is not stored as disk based extended table. If table is defined as extended table, then it is not stored as in-memory table. Data in in-memory tables are stored in the disk so that data can be recovered when power goes off / system is restarted.
Thanks for sharing
This option will help for growing entreprise data !
Thanks a lot for sharing on this imp. concept.
But when we separate warm data ex: some historical data on BW , in case of NLS where only reads are allowed, there is no need to write seperate BW query's to read the tables.
In case of dynamic tiering, since the data is stored on extended tables, we need to have a different BW query to read them. This requires a lot of rework or additional work. Do you have any comments on this .
A view can be created including (union) both in-memory and extended tables. You can use the view rather than in-memory and extended tables separately if the requirement is to access data from both stores.
Hi Sai,
This is a nice informative blog.
I have a question for you.
Is it possible to refresh just the dynamic tiering component from one system to another? Ideally I would like to know whether we can just do a system copy of the QA dynamic tiering server from prod.
SAP HANA in-memory tables and extended tables (part of dynamic tiering) are managed as single unified database. This means standard SAP HANA backup will back up both in-memory and extended tables together. That being said, you should be able to copy individual extended tables like any other SAP HANA tables.
Hi Sai,
Thank you for a very informative blog.
Could you help me with a question: Do we have NLS available for SAP Business Suite Powered by SAP HANA or is it just for SAP BW?
Looking forward to your insights,
Thanks.
NLS is available for SAP BW powered by SAP HANA and not for SAP Business Suite powered by SAP HANA.
Excellent blog!
Since your note on June 11, has SAP implemented Dynamic Tiering for SoH or S/4 HANA, or announced plans to implement it? Or is it still only BW?
Thank you!
Saiprashanth, thanks for explanations. I have following question around SAP IQ NLS and Dynamic Tiering technologies.
I have not migrated to HANA yet but I am looking at several options for this migration. I have a database of size 30 TB. Both, SAP IQ NLS and Dynamic Tiering technologies look completely different but serve kind of same purpose here that is “Reducing Size of Your Original Database”. From SAP IQ NLS it clearly indicate that I will remove data from my database and store that on separate server in compressed format. That means HANA cost for my database will be less because now my original database size is small.
With Dynamic Tiering, now I have a separate “Warm Store” that means by moving data into Warm Store my original database in “Hot store” will be smaller but still both Hot and Warm remains in same HANA database. Does that means my HANA license cost will go down (with warm store) also OR it will cost me same as 30 TB (not considering compression here)?
Thanks
Hi Sachin,
SAP NLS and Dynamic Tiering (DT) meant for different use cases. NLS is primarily an attachment to SAP BW for storing cold data (or read only archive). Where as DT is an attachment to HANA to store warm data. Warm data is an active data which your application modify just that it is less frequently accessed compared to data stored in HANA. Currently we support SAP BW and Native HANA applications for DT. DT is separately priced option based on data volume. Example: If your total data need 512GB of HANA for hot data attached with 3TB of warm data in DT, require two separate license costs for respective data volumes. Yes, DT helps you bring down the total costs of HANA by moving the data into DT layer where possible.
Hi Sai, is there any method, tool or technique which can be used to move older than 2 years data of S4/HANA to ‘tables in persistent storage’ from ‘in-memory tables’ and still be able to access data in both ‘tables in persistent storage’ and ‘in-memory tables’ through same single S4/HANA instance. I understand that Dynamic Tiering and NLS currently doesn’t support such concept for S4/HANA as mentioned above. If any other method, tool or technique is available, then please provide its details.
Thanks
Are there options to configure/designate Hot vs. Warm data? What criteria constitute hot and warm data? I presume that the SQL for Extended Table will be utilized for these configurations?
Need some clarification. When using Dynamic Tiering
Can I have a multi host HANA deployment without shared storage for Hot data nodes?
Can I use HANA System Replication as the HA/DR mechanism for HANA hosts that host hot data?
These questions come because our landscape does not have provision to provide shared storage for HANA Databases. We have NAS servers for /hana/shared file system but not for HANA data/log file systems.
If the above two are possible I am possibly looking at a landscape architecture that comprises of multi host HANA landscape with HSR as the HA/DR mechanism for HANA instanced that host Hot data and a single/multiple Warm hosts with Backup recovery as the DR mechanism.
Thanks
Naresh
Hello Sai,
To make it simple... with extended tables, we are treating certain tables as if they are on a disk based database system as opposed to in-memory DB. Blocks of data are loaded in to "data cache" to work on data and then changes are updated to disk etc. Whereas in in-memory case, data is loaded in to memory and update in memory. Savepoints save the data for persistence.
Is my understanding correct?
Thanks.
Hi , Thanks for the info . Its very helpful. Can we implement NLS with Sybase IQ(Cold) and HANA Dynamic tiering Service (Warm) both for SAP BW system on HANA. Can we install both HANA Dynamic tiering and Sybase IQ NLS implemntation on same host provided SAP BW on HANA is in another host. Pls let me know.
Hi, Thanks for the info. What is the case if I use Dynamic tiering without NLS for cold data. If we consider both Warm and cold data as warm data by using only Dynamic tiering. Pls let me know the use case for this. and another scenario, What if I use only NLS without using Dynamic tiering by considering both warm and cold data as cold data.