Skip to Content
There has been surge in the demand of sysIn Memory.jpgtem computing ability as more people have access to the applications with advent of smart phones and tablets. Data needs to be both real time and fast. In Memory Data management is one approach where we achieve the aforesaid objectives. But, the general question which everyone asks about In Memory is whether it is just a technology fad and whether it has any applicability to the modern day Enterprises.                                                      
1. In Memory Data Management is just a technology fad

Fact: It is the need of the hour

Data is growing exponentially. With more automation, more structured data is being captured in our organizations. Also, with the widespread adoption of social media, views ,expressions and opinions are getting captured in Face book status messages, twitter tweets, Instagram’s pictures etc. which are categorized as unstructured data. The explosion of data can be summarized with the fact that the humans have doubled the amount of data that they produced in the last century in the last decade.

Nimble Organization analyzes information at faster speed to stay ahead of their competitors. For Example, Smart organizations are capturing their success of their product releases by analyzing the tweets and messages of its products real time and even making necessary course corrections. To make smart and faster analysis of the dataset so large and complex (which have been named Big Data for their sheer volume), having a faster system performance has become a basic hygiene factor of any system.

Disk Scan speed has only become faster by 10 times over the last 30 odd years. Compared to the data growth, it is not adequate and will add to more scan time. There is a requirement to remove the redundancy involved in scanning and moving the data from disk to CPU for processing. In Memory Data Management removes two time consuming processes to make the processing faster – Scanning the data from Disk and moving it to the main memory (DRAM). As the CPU have direct access to DRAM, the processing power is enhanced exponentially resulting in desired efficiency.

2. In Memory Data Management Concepts are not applicable in Enterprises

Fact: In Memory Data Management concepts have evolved over the years to serve enterprises

Typical Relational Database Management System (RDBMS) stores the data in convenient way so that Data can be easily written on it but scanning these records is time consuming. Also, RDBMS cannot be compressed easily, a basic requirement, to load more data in DRAM. Using Columnar Database makes scan faster and enables more compression but with additional drawback of write back to Database overhead. Due to extensive encoding used for compression, it is expensive to insert, update and delete records in columnar database.

Enterprises end up using both the databases – RDBMS system to write to the database (OLTP Systems) and Columnar DB to scan the DB for analytics (OLAP systems). Data movement across the database not only adds overhead to the system but also makes the data dated.

In Memory Data Management has evolved to cater to the Enterprise Needs. For Example, HANA, the In Memory Data Management Platform, has the columnar feature but performs only insert operations and avoids the time consuming update and delete operations. It segregates the data into passive data (data which have infrequent use) and active data ( regular/ recent data) for storage and operation optimization. Similarly, relevant modifications are done in the In Memory Data base to make it optimal for enterprise usage.

In Memory Data Management is the answers to today’s enterprise growing data computing demands. If optimized well, as seen in HANA, it can deliver amazing results such as  analyzing  F1 race at real time to de-codify the genome project much faster than before. Also falling memory prices and availability of additional processing power make In Memory Data the most suitable technology to invest for future. Would love to listen to your comments on this article.

To report this post you need to login first.

4 Comments

You must be Logged on to comment or reply to a post.

  1. Arne Weitzel

    Good post! I do believe though that you can have any database in-memory just by configuring a cache and using SSD instead of usual disk storage. The point is more that databases such as Hana can compress the data much better by using columnar storage. This way the database size decreases very much and so you can afford to put into more expensive memory / SSD.

    (0) 
    1. Subhankar Pattanayak Post author

      Thanks for the feedback. Absolutely, i totally agree that encoding plays a critical role for any In Memory Data Base Design. I had intentionally not mentioned it to keep the blog simple to read for users.

      (0) 
  2. Manas SAHOO

    I agree with you Subhankar. In today’s world leveraging technologies like HANA to manage huge data volumes and to get real time insights is no longer “nice to have” but has become “necessary”. If you dont leverage the trends in technology, then soon your relevence may be questioned (remember the “photo film” days, the “gramaphone records” days). And those that do create differentiation in market. You may like to look at my blog on this topic at http://scn.sap.com/community/hana-in-memory/blog/2012/11/14/who-says-hana-gives-just-technical-benefits-and-interests-only-the-it-team

    and other customer stories at http://www.saphana.com/community/customer-stories

    (0) 

Leave a Reply