With the release of SAP HANA 2.0 SPS 02, HANA dynamic tiering has introduced some exciting innovations:
Adoption of dynamic tiering is on an upward trend! Customers have begun incorporating dynamic tiering into their HANA systems to manage ever increasing volumes of data – keeping active data in HANA memory, and moving older data to dynamic tiering. With multistore tables, data modeling is not impacted by adding the dynamic tiering disk store, allowing for quick implementation, and a seamless experience. Enhanced query optimization, results caching, and improved efficiency in cross-store data transfer options have elevated dynamic tiering performance to levels that are meeting and exceeding customer expectations.
I want to share with you a compelling customer implementation that demonstrates how HANA dynamic tiering is delivering on the promise of low TCO warm data management for native HANA applications.
In this scenario, a customer in the electrical and gas utilities industry has implemented a billing application in HANA. Their billing application queries large volumes of meter data, to measure how to properly bill users for energy resource consumption. The HANA system is managing about 24TB of meter data, and is configured as a 5 node scale out set up, with an additional dynamic tiering (DT) node to handle older data that does not need to reside continuously in memory:
The HANA machines are powerful – each with 144 cores (288 threads with hyperthreading), and 6TB of RAM. The dynamic tiering server, by contrast, is running on much smaller commodity hardware – 72 cores (144 threads with hyperthreading) with 512GB of RAM.
The database tables for meter data usage are implemented as multistore tables. Multistore tables are HANA partitioned column tables with some partitions residing in HANA memory and some partitions residing in the dynamic tiering disk layer. The billing application is highly parallelized with 999 concurrent threads performing processing. Each thread reads the multistore tables, working on independent subsets of primary keys. A billing run with all data in HANA was executed as a baseline, then 25% of the older data – approximately 6TB - was moved to dynamic tiering. (The ALTER TABLE…ALTER PARTITION SQL statement was used to move partitions of data from HANA to dynamic tiering.) After the data was moved, the billing run was executed again. Here are the performance results:
With 25% data moved to dynamic tiering, running on a relatively low cost commodity machine, the performance of the billing application was reduced by only about 10%.
This is a very impressive result! The customer has freed up 25% of HANA memory for hot data processing, has moved off older data to a cheaper processing tier, and is still achieving excellent performance – within 10% of pure HANA.
To learn more about dynamic tiering, check out our HANA dynamic tiering developer page: https://www.sap.com/developer/topics/hana-dynamic-tiering.html
- Multistore tables for partitioning data across HANA memory and dynamic tiering disk
- Intelligent “results caching” to significantly improve performance of cross-tier SQL join operations
- Support for HANA system replication for high availability and disaster recovery
Adoption of dynamic tiering is on an upward trend! Customers have begun incorporating dynamic tiering into their HANA systems to manage ever increasing volumes of data – keeping active data in HANA memory, and moving older data to dynamic tiering. With multistore tables, data modeling is not impacted by adding the dynamic tiering disk store, allowing for quick implementation, and a seamless experience. Enhanced query optimization, results caching, and improved efficiency in cross-store data transfer options have elevated dynamic tiering performance to levels that are meeting and exceeding customer expectations.
I want to share with you a compelling customer implementation that demonstrates how HANA dynamic tiering is delivering on the promise of low TCO warm data management for native HANA applications.
In this scenario, a customer in the electrical and gas utilities industry has implemented a billing application in HANA. Their billing application queries large volumes of meter data, to measure how to properly bill users for energy resource consumption. The HANA system is managing about 24TB of meter data, and is configured as a 5 node scale out set up, with an additional dynamic tiering (DT) node to handle older data that does not need to reside continuously in memory:
The HANA machines are powerful – each with 144 cores (288 threads with hyperthreading), and 6TB of RAM. The dynamic tiering server, by contrast, is running on much smaller commodity hardware – 72 cores (144 threads with hyperthreading) with 512GB of RAM.
The database tables for meter data usage are implemented as multistore tables. Multistore tables are HANA partitioned column tables with some partitions residing in HANA memory and some partitions residing in the dynamic tiering disk layer. The billing application is highly parallelized with 999 concurrent threads performing processing. Each thread reads the multistore tables, working on independent subsets of primary keys. A billing run with all data in HANA was executed as a baseline, then 25% of the older data – approximately 6TB - was moved to dynamic tiering. (The ALTER TABLE…ALTER PARTITION SQL statement was used to move partitions of data from HANA to dynamic tiering.) After the data was moved, the billing run was executed again. Here are the performance results:
- All data in HANA: 5M records processed per hour, for a duration of 53 minutes
- 75% of data in HANA and 25% of data in dynamic tiering: 4.6M records processed per hour, for a duration of 58 minutes
With 25% data moved to dynamic tiering, running on a relatively low cost commodity machine, the performance of the billing application was reduced by only about 10%.
This is a very impressive result! The customer has freed up 25% of HANA memory for hot data processing, has moved off older data to a cheaper processing tier, and is still achieving excellent performance – within 10% of pure HANA.
To learn more about dynamic tiering, check out our HANA dynamic tiering developer page: https://www.sap.com/developer/topics/hana-dynamic-tiering.html
- SAP Managed Tags:
2 Comments
