Harnessing Data Explosion
Part 1 of the 6-part ‘Unleashing Self-Service Business Intelligence’ series.
When it comes to data, there’s a lot of it! Data is being produced at astronomical rates. In fact, 90% of the data in the world today was created in the last two years! It is changing our world completely and shows no signs of slowing. The term “big data” can be defined as data that becomes so large that it cannot be processed using conventional methods. The size of the data which can be considered Big Data is a constantly varying factor and newer tools are continuously being developed to handle this Big Data.
Big Data can take many forms. It can be structured or unstructured. It can live in databases, data lakes and on IoT sensors. And it can be historical or streaming. What does all of this mean?
Historical data is generally defined as collected data about past events and circumstances pertaining to a particular subject. Historical data includes most data generated either manually or automatically within an enterprise.
Streaming data is data that is generated continuously by thousands of data sources, which typically send in the data records simultaneously, and in small sizes. Streaming data can include a wide variety of data such as log files generated by customers using mobile or web applications, ecommerce purchases, in-game player activity, information from social networks, financial trading floors, or from connected devices and sensors.
In order to make sense out of this overwhelming data explosion, it can be broken down into ‘the Big Data Vs’ During this blog series, we’ll discuss the growing importance of self-service BI and data visualization, and we’ll take a closer look at 4 of the Big Data Vs: Variety, Velocity, Volume, and Value.
SAP VP Tom Roberts recently spoke with Nick Halsey, CEO of Zoomdata, a provider of modern analytics for big and fast data, to discuss this explosion of data, and how technology is advancing to support interactive analysis of billions of rows of varying types and sources of data. Nick observed, “This explosion of data, this world of IoT, is changing everything. And old (business intelligence) platforms aren’t designed to be real time.”
See the first of their 6-part discussion.
Learn more about data visualization.