In this blog post we will see the options are available for a SAP and non-SAP business system for Data Management.
To manage large volumes of data a typical customer usually thinks of 2 options for the database server (i) scale-up (ii) scale-out. However There is always a chance of DB reaching a point to where scale up OR scale out is not possible due to hardware/software limitations. The customer is often left with a question as what can now be done with the “old” data. Also there could be some legal requirement to retain the data for a longer duration of time.
A typical scenario where a database for business system grows beyond certain limit definitely calls for a proper data management which helps the customer to not only get rid of ‘old’ data from the active main database but to also keep it in such a way that is accessible on demand. Also more companies are opting for a multi-cloud approach which means that even the data is not localised in one server or data centre or geographical region. Future business scenarios will make use of new technologies like Machine learning. AI , IoT which demand very large volumes of data to be stored and accessed across locations.
A future data management strategy should help customers operationalise such large and scattered data sets. Data should be reliable and well connected at the same time provide flexibility to the end users to access the data in reasonable amount of time.
Data Management (Data Aging | Data Archiving)
Data Management has evolved a great deal and there are various options available matching several use cases.
For a given business system, based on the customer scenario there could be different approaches here. These approaches also depend on the type of application being used.
Let’s start by first differentiating OR understanding Data Archiving and Data Aging.
Data Archiving is a popular method available to effectively manage the data especially for SAP Business systems and there are many approaches to do archiving. The solutions are basic and read operations over the archived data are very slow in terms of I/O speed.
Usually all the data isn’t required for day to day operations, but organisations simply cannot delete the organisational data due to legal compliance issues. SAP Information Lifecycle Management provides various possibilities for the customer to archive the old data. (SAP Information Lifecycle Management).
The key capabilities of SAP information Lifecycle Management are listed below.
- System decommissioning
- Retention management
- Data archiving and management
- Manage data volumes while keeping business context completely intact
- Move outdated data securely into long-term, less expensive storage
- Provide convenient access to archived data.
Above picture shows classic archival storage in ERP and BW
Data archiving helps customer to reduce the data footprint by isolating very old data however it alone cannot solve the problems and challenges of the modern business scenarios.
Data Aging offers you the option of moving large amounts of data within a database so as to gain more working memory. You use the relevant SAP application to move data from the current area to the historical area. You control the move by specifying a data temperature for the data. The move influences the visibility when data is accessed. This means that you can perform queries of large amounts of data in a much shorter time. To be able to apply data aging to your data, you need to fulfil certain requirements regarding the database and the application.
The Data Aging framework is available with NetWeaver release 740 SP05 onwards. SAP HANA supports Data Aging. The type of data storage now depends on the frequency of the data access, data is classified as Hot-Warm-Cold. At any moment in time all the data is accessible hot-warm-cold and the time to fetch the data increases as the data temperature drops.
SAP provides different solution for storing aged data. Data stored in different tiers according to the temperature of the data.
With SAP HANA in memory Hot data resides in the main memory, for warm data there are solutions as Dynamic Tiering(for native HANA solutions, Standalone Datamart scenarios) / Extension nodes and the newly introduced SAP HANA Native Storage Extension. For cold (for SAP BW on HANA and BW/4HANA systems) / NLS and ILM.
When using HANA as a Database new SAP tools like SAP HANA Data Warehousing Foundation – Data Lifecycle Manager customer can manager their data E2E.
When it comes to HANA based systems then we have pre-defined options available for Data management. Below picture gives an example.
Below table provides the current options available for data management for standard SAP solutions.
It is important to note thatSAP Data archiving & SAP Data Aging are techno-functional topic, the actual placement of what data sits in which tier and how much data sits in each tier is always decided and controlled by the customer. Functional expertise may be required here and in-house consultants from a customer should also be a part of any such projects.
- Reduced data footprint means better performance, reduced backup & restore times.
- Leaner systems mean that the migration of such system to cloud is easier and faster.
- Faster adoption of new SAP products like SAP S/4HANA & SAP BW/4HANA.
- SAP HANA provides the perfect playground for modern data management solutions.
- Usually the cost of storage for Archived data is less as compared to the regular storage.
High level decision chart for Data Management