Big Data is the term currently being used to describe the situation a company faces when the size and/or complexity of their data make it difficult for traditional data management technologies to handle.

In recent months, the Big Data topic has been covered in print and online by both business and even consumer publishers.

A Few Big Data Examples

• Weather conditions (temperature, wind speed, humidity, pressure), taken every second, from every weather monitoring station on the globe

• DNA sequencing of human and other genomes

• Consumer data (what products are sold, at what price, and which location) from every store in the world.

• Energy consumption each minute, from every meter, from every building and home in the world.

The Three Vs

As early as 2001, Gartner analyst Doug Laney defined the three major problems created by the explosion of information in the Big Data era – Volume, Velocity, and Variety (Three V’s).

• Volume – the sheer amount of data being created doubles the amount of available storage, and 95% of this data is complex or unstructured in nature.

• Velocity – data has become a strategic asset being used as a basis for competitive advantage; the faster a company can access, manipulate, and gain insights on its data, the better.

• Variety – there are countless sources and types of data today; moving outside of the corporate firewall with sensors, social media, photos, and video to name a few.

These Big Data problems have led to an influx of modern, alternative data management technologies and startups, each with the goal of solving one or more of the Three V’s. Companies are entering the market at three different layers of the technology “stack” – Database, Enterprise Information Management (EIM), and Business Intelligence (BI)/Analytics.

The Driving Forces Behind Big Data

The Big Data phenomenon materialized from a perfect storm of social, technological, and economic forces that came together in a very short period of time. A few that are worth pointing out are:

• Networked World – embedded sensors are everywhere, from smart meters to automobiles to refrigerators; these sensors are producing continuous streams of data in real-time.

• Mobile – there were over 5 billion mobile devices in use in 2010; mobile users are 70% more active on Facebook and Twitter than non-mobile users.

• Economies of Scale – the price of storage is falling while capacity is increasing; data centers are being built all over the world to help companies monetize their data deluge.

According to SAP’s Gurdeep Dhillon (@gurdeepd)  “An opportunity exists to re-focus the market definition of Big Data on what is truly most important to customers. Solutions which merge external, unstructured ‘social’ data with transactional ‘business’ data and deliver valuable insights at speeds which were previously unimaginable are the next big competitive differentiator.”

In-Memory Technology

In-memory computing gives organizations a means to manage the rapid growth in data volume and complexity, while at the same time, exploiting that data to achieve quicker business insight. With the ability to process hundreds of billions of rows of data in seconds, in-memory technology enables business leaders to make quick decision at the speed of thought, spark deeper insights and offer instant answers to multiple scenarios within seconds.

Learn More About Big Data…

• To learn more about Big Data, check out this article on how to achieve the benefits of a real-time business.

• Read IDC’s predictions about the future of In-Memory Computing.

• Watch this video on the business value of real-time analytics.

And subscribe to our updates on the latest innovations in Big Data.