The Timely Value of Data
Have you ever pulled into the drive-through of your favorite fast food restaurant to find a long line of cars in front of you? In that situation did you think to yourself, “I would be glad to pay a little more not to have to wait in this long line?” I know I have – many times. Here’s the point: we all experience the perceived need for instant gratification. It is a common trait in our modern culture along with the willingness to pay more to meet that need in a timely manner.
In the realm of data warehouse architecture, this same phenomenon is referred to as the Principle of the Timeliness of Data. Simply put, this principle states that the value a user attributes to data directly correlates to the user’s access time requirement for that data.
Consider an arbitrary Sales Orders dataset. The speed of access to new Sales Orders or even orders from this year must be of the highest priority. Operational decisions and insights are critical to business processes. However, slower access to sales orders from ten years ago, as an example, rarely imposes on the immediate needs of the business. The truth here is that the query response expectation is not as critical for data as it ages over multiple years – most users are fine just to have access to that data.
Data storage utilizing in-memory database technology like SAP HANA places this principle right at the center of a proper data architecture. To explain further … critical data should reside only in memory using column tables. Consider this data to be your hot data. To save memory storage space, one does not need to load the whole table into memory at the same time – just load the individual columns. For slightly slower access requirements (think warm data), SAP HANA can load pieces of these columns a page at a time. The trade-off is performance for more storage space. Finally, SAP HANA can even satisfy the coldest of data access requirements utilizing disk space storage.
The reason most seasoned disk-based data warehouses now struggle to meet the needs of today’s data-hungry consumer is that multiple copies of the data are increasingly necessary to maintain performance. Hindsight is always 20/20 … Ultimately, the lesson a data warehouse administrator realizes over time is that change is manageable during the early years, but inevitably the compounding complexity from data duplication hinders the ability to respond to business requirements.
Remember the crowded drive-thru from earlier? A data warehouse that uses disk-based storage for hot data may be the source of the same type of consumer frustration, dissatisfaction, and even a desire “to go get it somewhere else.” Just like no one wants cold food when expecting hot food, the business can no longer wait for data they need now.
I encourage you to seriously consider migrating your data warehouse to SAP HANA – the most advanced in-memory database technology. Think about the bottlenecks that you can now overcome by combining in-memory performance with a multi-temperature architecture. With only one single copy of the data residing in the right tier, you now can establish a foundation for innovation for the current and future needs of your business …
So let’s stop serving cold data in the data drive-thru. Learn more about SAP HANA and even put hands-on today. Your business users will thank you for the great experience – timely hot data delivered every time.