“Big Data” is a term used to describe massive information stores – generally measured in petabytes and exabytes – and also refers to the methods and technologies used to analyze these large data volumes. The core principles of Big Data (data mining, analytics) have been around for some time, but recent technology has enabled the collection and analysis of previously unimaginable data volumes at extremely high speeds.
The enormous utility and potential of Big Data is well-appreciated by businesses, government, and law enforcement (and hackers and identity thieves), but it’s not well-conceived by the average person.
That may be about to change. For good or ill, the impact of Big Data in our daily lives is growing. Here are just a few examples of how Big Data plays a role in everyday activities:
How we spend
To understand customer buying patterns, retailers rely on the somewhat myopic datasets derived from club-card and loyalty-points programs. To get customers “in the door” they offer mass, deep discounts on “special” shopping days, like black Friday. Big Data is changing all this, enabling retailers to capture, store, and analyze more, and more diverse, data on their customers and the competition. As a result, marketing is becoming more targeted and shopping experiences much more personalized. Real-time offers tailored to where you are and what you tend to buy, and when you tend to buy it, will soon be commonplace. Already, some online retailers display different prices to different users, based on their purchase history and other data.
How we vote
In 2012, Big Data made, arguably, its first major splash outside the business world, playing a role in the U.S. presidential election. President Barack Obama’s campaign ran what has been referred to as the first Big Data-powered presidential campaign. Instead of using traditional polling – extrapolating from small samplings to gauge the sentiment of voters – the Obama camp collected and analyzed data from huge numbers of actual individuals. This was historic because it upended traditional methods of running campaigns. Mounds of data from surveys, phone calls, external voter lists, and past voting patterns drove real-time voter outreach and get-out-the-vote efforts.
How we study
A number of academic institutions are employing Big Data to address high dropout rates. New technology is enabling school officials to use data – such as standardized test scores, grade histories, and even real-time statistics about what a student clicks on during online classes – to create an individualized academic profile for each student. By applying statistical models to those profiles and measuring them against those of similar students, schools can identify likely outcomes – such as successfully completing a class or major. This helps school officials steer students toward majors and classes in which they are likely to succeed.
How we stay healthy
Healthcare has historically been a difficult domain for analytics, as privacy and regulatory restrictions greatly restrict or outright forbid the use of general patient data in research. The proliferation of smartphones and other medical “self-tracking” devices is, however, changing the landscape. It’s now possible to collect 24/7 data from healthy individuals via these devices, creating a large, unbiased control group that can be segmented by demographics. Insight derived by analyzing these data volumes can help people make healthy lifestyle choices, take preventive measures (e.g., flu vaccinations), and predict their chances of being afflicted with certain diseases. And real-time personal analytics on how a person’s daily behavior impacts their health are likely not far off.
How we safeguard (or forfeit) our privacy
The potential benefits of Big Data are real and compelling, but so are the risks – among them privacy. The saying goes: You’re not paranoid if someone’s actually after you. The collection, analysis, and sale of personal data – from your shopping habits to which health issues you Google – is a booming business. And an increasing amount of personal data – particularly online and on smartphones – can be, and is, collected without the user’s knowledge or consent.
Still, despite well-founded fears that Big Data could morph into Big Brother, most data collected today is shared willingly. As citizens and consumers become increasingly aware of the impact and proliferation of Big Data, the savvy among them will abandon the ‘ignore-the-man-behind-the-curtain’ strategy in favor of a practical understanding of how data about them is gathered, managed, and used.
Follow Siddharth Taparia on Twitter: @siddharth31