Technical Articles
SAP BW/4HANA Data Tiering Optimization
Dear friends of SAP BW/4HANA,
I’m working for SAP since more than 10 years and as a member of the BW product management team I received a lot of questions from customers regarding new functionalities for data lifecycle mangement and data aging strategies in SAP BW/4HANA.
Therefore, I would like to share a summary of the currently available information.
With SAP BW/4HANA 1.0 SP04 SAP is providing functionality to classify persistent BW data into hot, warm and cold.
In this blog, I’ll explain the advantages of the SAP BW/4HANA exclusively available new feature for Data Tiering Optimization (DTO) in technical details.
Overview:
Data Tiering Optimization helps SAP BW/4HANA customers to classify the data in the DataStore object (advanced) as hot, warm or cold, depending on the cost and performance requirements for the data.
Depending on this classification and how the data is used, the data is stored in different storage areas. Data Tiering Optimization provides a central UI, where all storage options can be set. The partitions from the SAP HANA database are used for this.
The following options are available:
- Standard Tier (hot): The data is stored in SAP HANA.
- Extension Tier (warm): The data is stored in a SAP HANA extension node. Only SAP HANA Extension Nodes can be used for warm data storage in SAP BW/4HANA – SAP HANA Dynamic Tiering is not applicable to SAP BW/4HANA.
- External Tier (cold): The data is stored externally (SAP IQ or Hadoop (Hadoop is only available with SAP BW/4HANA 1.0 FP08 or higher)).
Overview presentation:
SAP Data Tiering Optimization is an option to optimize the memory footprint of data in SAP BW/4HANA and streamline administration and development, thereby reducing TCO.
Introducing one data tiering concept for hot, warm and cold data based on Advanced DataStore Objects enables users to create full flexible partitions based on the data temperature in SAP BW/4HANA. Find the full presentation here:
Data Tiering Optimization with SAP BW/4HANA – Updated Version
Furthermore find a technical overview presentation of the SAP HANA Extension Nodes here:
SAP HANA Extension Node – Technical Overview presentation
Getting started
General Information
- SAP Help Portal: Data Tiering Optimization in SAP BW/4HANA
- SAP Note 2517460 – SAP BW/4HANA Data Tiering Optimization: Information, Recommendations and Limitations
Set Up and Installation
Cold Store with SAP IQ (fka Nearline Store)
- SAP First Guidance – SAP BW: Implementation SAP-NLS/SDA with SAP IQ16.0
- Please note that the respective Remote Source (HANA SDA) to IQ (ODBC) needs to be “readwrite” enabled
- SAP Note 1796393 – SAP BW near-line solution with SAP IQ
- End-to-end Document — SAP IQ as a Near-line Storage Solution for SAP BW
- Configuring SAP IQ as a Near-Line Storage Solution
- SAP IQ 16 Hardware Sizing Guide
Warm Store with SAP HANA Extension Node
- SAP Note 2343647 How-To: Configuring SAP HANA for the BW Extension Node
- SAP Note 2453736 How-To: Configuring SAP HANA for SAP BW Extension Node in SAP HANA 2.0
- SAP Note 2415279 – How-To: Configuring SAP HANA for the SAP HANA Extension Node
General Information
- SAP Note 2296290 – New Sizing Report for BW/4HANA
Implementation and Administration
Cold Store with SAP IQ (fka Nearline Store)
- SAP Note 2165650 FAQ: BW Near-Line Storage with HANA Smart Data Access
- SAP Note 2100962 FAQ: BW Near-Line Storage with HANA Smart Data Access: Query Performance
- SAP Note 1999431 SIQ: Setting up SSL for connections to IQ
- SAP Note 2133194 Can SAP IQ run in a cloud environment?
Warm Store with SAP HANA Extension Node
- FAQ: SAP BW/4HANA and SAP BW-on-HANA with SAP HANA Extension Nodes
- SAP Note 2486706 – FAQ – SAP BW/4HANA AND SAP BW-ON-HANA WITH EXTENSION NODES
- SAP Note 2643763 – FAQ – SAP HANA Extension Node for SAP HANA native applications
- SAP Help Portal: Using an Extension Node for Warm Data
- SAP Note 2334091 BW/4HANA: Table Placement and Landscape Redistribution
- SAP Note 2317200 Data lifecycle management for BW on SAP HANA and extension nodes
Nice job Gordon, I sent a few questions to you via email. Thanks!
Hi Gordon,
What does "Dynamic Partitioning Schema" mean ? It is mentioned in "Future direction". Doesn't it contradict when we say "SAP HANA Dynamic Tiering is not applicable to SAP BW/4HANA."
Or the is a mistake in my understanding.
Regards,
Mihir Kiri
Hi Mihir,
“Dynamic Partitioning Schema” means that the partition is able to cope with additional data outside of the expected range and DTO is able to self extending partitioning schemes.
Furthermore for warm data in DTO we are using Extension Nodes (additional Hana Scale Out node with a relaxed sizing formula and RAM/CPU ratio. Therefore SAP HANA Dynamic Tiering is not applicable and not supported with SAP Data Tiering Optimization.
BR Gordon.
Excellent update for BW4HANA toolset to maintain this directly at BW4HANA instead manual maintenance at different levels!! Then, SAP HANA landscape reorganization was replaced by Data Distribution Optimizer for BW4HANA, doesn't it?
From LSA++ perspective, How could be manage effectively Corporate Memory for huge aDSOs when handled in an scale out architecture, where exists multiple tables bigger than 2 billion recs per table and with non cumulative key figures? I understood that must be partitioned (with Cold data retained at SAP IQ) and also used Semantic group (also applying multitemperature concept once it was classified by a key field (region, country, etc) but still I have restriction for non cumulative key figures at SAP IQ and maximum partitions are limited to 1024.
Please let me know
This is the best introduction to DTO, technical prerequisites and usage!
Also, in the latest update from November 2018 I think one limitation is more clearly mentioned:
*Validation with upcoming Vora release still pending (see note 2608405 SAP Vora cold store: Informations, Recommendations and Limitations ).
Thanks for this DTO overview! I understand that data tiering is only possible in aDSO's. Can it be applied in aDSO's in "BW on HANA" or is it only possible in BW/HANA? When I check the aDSO in our system (7.5 SP11), I only see the checkbox "SAP BW Data Tiering" and no setting for temperartures... Is there documentation available for temperature settings when using "BW on HANA"...
Hi,
DTO is exclusively available for SAP BW/4HANA.
BR
Gordon.
Hello Gordon
Congratulations on this blog, it is very helpul.
I have a question for you, I'm following the steps of the video Build a aDSO and creating partitions for hot, warm and cold data. but I'm getting this message (see image below)
Maybe I'm doing something wrong. What I'm trying to do is, to split a data set in 3 different months (by partionting object) and then apply the corresponding data temperature to each month of data.
This is the first time I'm doing this, and probably I made a mistake or something. Could you guys help me in finding what am I missing?
Best regards,
Miguel.
Hi Miguel,
You want to change a DataStore object (advanced) that contains data.
The System created a Remoling Request. Please execute the remodeling request in the backend.
More Information here: https://help.sap.com/viewer/107a6e8a38b74ede94c833ca3b7b6f51/1.0.8/en-US/53054b5f721149c19136e71e30d44dab.html?q=remodeling%20adso
BR
Gordon.
Hello.
Thanks a lot Gordon! I have one last question. I clicked on the "remodelling is pending" message, and it took me to the RSCNV_MONITOR transaction. But I don´t see the remodelling steps there...
I have serious doubts about this, I mean, am I doing something wrong? or maybe is there some step that I am missing?
Do you recommend me to open an expert chat, maybe?
Greetings!
Hi,
yes you should see the remodeling request and the steps there. Have you tried to Start the Request (top Right).
Otherwise please open a support ticket.
BR
Gordon.
Thank you so much Gordon.
Hi,
You can go to same screen from RSMONITOR t-code and in the selection ensure you select the from date as 'on or before the date on which the change is moved/done to that ADSO', then you should find that request.
In case you have deleted the data in the ADSO then re-modelling request will be automatically removed some times.
Regards - Avinash.
Hi Gordon
Very useful blog, Thank you.
I have a question on the hot, warm and cold tiers, what is best practice for use of these tiers, I have read through documentation but I am still not clear.
Many thanks
Steve
Hi Gordon,
are there any plans to make the new HANA Native Storage Extension (NSE) from HANA 2.0 SPS4 available as a further option for warm storage in DTO (for customers having no scale-out)?
If yes when and from which BW release?
Many customers are looking for a solution to reduce memory consumption without adding too much complexity to the architecture...
Thanks
Christoph
Well I think here is the response: NSE is available from BW/4HANA 2.0 SP 4 onwards:
See: What's New in SAP BW/4HANA 2.0 (Release Information) 2.0 SP4 => Operation =>
Data Tiering Optimization (Enhanced)
https://help.sap.com/viewer/b3701cd3826440618ef938d74dc93c51/2.0.4/en-US/5d4fb4c79e3e4acabd068bc0c302c810.html
Hello,
Thank you for this article.
I have a question. We are on BW4HANA 2.0 SP1 and have IQ configured as NLS.
Through DTO, I am able to change multiple partitions to Cold store. But when try to get them back to Hot, it gives exception. One at a time is working fine.
Is this a limitation, or a configuration bug?
Hi Gordon,
Thanks for nice article.
I have seen the video "Creating a query based on the aDSO partitions.", however for reading near line storage there is an option of creating a manual input variable which we used in old BW. Seems like same option is available in BW/4HANA but I am not able to find that create button.
Can anyone help me (guide me) on how to create a nearline storage variable in BW/4HANA, so that option/choice can be given to end-user to choose.
Thanks & Regards,
Avinash.
Hi Gordon,
I do have a customer who is interested in NLS. We discussed this and two Questions came up:
also be maxDB instead of IQ-DB. Admins would prefere maxDB as they are not familiar
with the IQ-DB at all.
Is it for free when we use it for BW-Embedded NLS and if not who to calculate the Costs then?
Really would appreciate your help on this. Thank you.
Kind regards,
Klaus
Hello Klaus Reiner
for NLS in general, see -
Note 2780668 – SAP First Guidance – BW NLS Implementation with SAP IQ
see also the Blog - SAP (Sybase) IQ – the hidden treasure …
for Details, see https://blogs.sap.com/2016/10/12/sap-nls-solution-sap-bw/
These Answers cam from Roland Kramer
Best Regards Roland Kramer