Earlier this month, Typhoon Haiyan—the most powerful storm to ever hit the Philippines—left thousands dead and many more without food and shelter. Luckily for many, big data came to the rescue.
NASA researchers Michael Cox and David Ellsworth coined the term “big data” in 1997 as they tried to process and visualize the vast amounts information generated by supercomputers.
Today big data is still a big deal, but it is generated more by mobile phones than by computers and has become easier to use as an analysis and predictive tool. Big data has become a vital resource for companies eager to understand their customers—and, perhaps more importantly, practitioners in fields such as public health and humanitarian relief are using big data as a critical driver of empirically-based problem solving.
The death count in the Philippines from Typhoon Haiyan is expected to reach 10,000, which is unprecedented for the storm-plagued country. While this is a colossal loss, many Filipinos in vulnerable areas were saved thanks to big data analysis that enabled rescuers to reach them just in time.
One of the major problems faced by rescue teams during the crisis was sifting through a barrage of confusing and conflicting reports. Social media sites made matters worse by giving fire to wild rumors exaggerating damage and number of deaths.
Action was required and time was running out for survivors. Patrick Meier, co-founder of Harvard’s Humanitarian Initiative, dealt with the problem by using an innovative approach called social mapping. Using a combination of algorithms, he provided rescuers with a detailed, data-driven map of the areas they should target first and the quickest ways to reach there. All of this was done by analyzing thousands of tweets for certain keywords, such as appeals for help, reports of damage, or shortages of medical supplies. Among other factors, the United Nations is relying on Meier’s data to firm up its own massive rescue efforts.
Disaster Relief International (DRI), which has provided over $1.4 billion of privately funded humanitarian aid worldwide since the year 2000, is using big data analysis to improve disaster response efforts in the Philippines by determining where help is needed most and using real-time tracking of assets and personnel, all at the click of a button. Responders are using GPS-enabled satellite communicators for health and structural needs assessments. DRI has implemented SAP solutions and the SAP NetWeaver platform to help organize its inventory and relief efforts worldwide. Last year, DRI was awarded the Peter F Drucker award for Non-profit Innovation.
Such uses of big data are relatively new, and they are being driven by partnerships between the public and private sectors. The first wide usage of big data for driving relief efforts was in 2010 during the devastating Haiti earthquake. Scientists from Sweden’s Karolinska Institute used data provided by Haiti’s biggest cellphone operator, Digicell, to compare people’s movements before and after the earthquake and predict hotspots where medical supplies were needed to prevent disease from spreading.
SAP partnered with UNFPA in 2011 to create a dashboard for analyzing information related to the world’s population reaching 7 billion. Using the dashboard you can see, for instance, that 21 percent of Pakistan’s 42 million children are enrolled in secondary school. And out of these, 43 percent are girls. This can be powerful information for global public policy makers.
These are just some examples of how big data has become more open and organizations have become more willing to share anonymized metadata. Orange, the mobile operator, opened up its call records metadata last year to big data developers in its annual D4D Challenge. Several scientists took advantage of the opportunity to do innovative analysis. They found ways to predict the next outbreak of malaria in Kenya. Another group designed a new transport system for the traffic-clogged cities of the Ivory Coast that will save travel time by over 10 percent. Almost all of the research was about finding ways to improve society.
All of this is leading to democratization of big data, assuaging concerns about emergence of data monopolists. Thanks to sharing of information by companies and government agencies, anyone can access and analyze big data. From helping save people in the Philippines to the prevention of deadly diseases in Africa, big data has become a force for change. Quite a journey from the days of Cox and Ellsworth.