Big Data in Practice – What Can We Learn from Successful Organizations?
Our blog today is contributed by Dr. Carsten Bange, founder and managing director of the Business Application Research Center (BARC)
The term “Big Data” is often used as an umbrella for initiatives or strategies to integrate, store, and analyze data from many internal and external data sources in heterogeneous formats – and most importantly to gain benefits from all these activities. But how do companies actually use it and what are the benefits achieved and challenges faced? We asked more than 550 professionals in companies worldwide several questions about their Big Data initiatives. Here are some of our key findings and resulting takeaways.
Big Data Usage and Drivers
The use of Big Data is increasingly prevalent in today’s enterprises. According to our survey, relatively few companies (17%) would not even consider starting a Big Data initiative. On the contrary, over 40% already have experience with Big Data, whether as an integrated part of their business processes or as a pilot project. In companies where Big Data initiatives are already an integrated part of business processes, management is the main driver or thought leader. Line-of-business managers, on the other hand, often are more passive and much less likely to be the drivers behind this topic.
Takeaway 1: While top management is often the driver around Big Data initiatives, middle management buy-in and an understanding of the value of data across all business units evolves as a key success factor.
Organizations are looking to address a wide range of challenges through their Big Data initiatives. Companies planning to start a Big Data initiative most frequently cited the following reasons for doing so: analysis of large data volumes (57%), analysis of data from different sources (50%), faster and better analysis (55%), and more sophisticated forecasting methods (51%).
Takeaway 2: Big Data is all about analyzing new and large amounts of data to gain actionable insight. Advanced and predictive analytics play an increasingly important role in unlocking the value in data.
38% of respondents are still searching for compelling use cases. Moreover, data privacy, security, and a shortage of Big Data expertise are other major problems facing enterprises when dealing with Big Data. 49% of respondents named data privacy and 48% cited data security among their biggest problems in using Big Data technologies.
Takeaway 3: Many Big Data use cases center on customers. Organizations need to take special care in protecting their data and define data privacy rules. As the use of Big Data analysis spreads, the complexity of the technological infrastructure tends to grow as well. Securing both this infrastructure and its data are central challenges for companies.
The second main issue with Big Data is the lack of internal business and technical know-how for tapping and analyzing Big Data. 53% of respondents cited a lack of analytical know-how, while 48% reported a lack of technical expertise in their organizations. 30% of companies with Big Data projects, therefore, want to create new jobs in these areas. The current global labor market, however, can’t meet demand for this type of expertise.
Takeaway 4: It’s important to both attract and retain qualified personnel with analytical/statistics, data management, and also domain know-how. Only the combination of these three skills will lead to successful Big Data projects. Technology and business applications move quickly, so constant learning is vital. Training existing staff on Big Data capabilities is important since skills are rather scarce in the current job market.
The good news is that in many cases where organizations have started to use Big Data, projects have not only met with companies’ high expectations, but have often exceeded them with even better results. Topping the list of benefits achieved through Big Data initiatives are better strategic decisions (69%), improved control of operational processes (54%), better understanding of customers (52%), and cost reductions (47%). This shows a high divergence in expectations but also benefits achieved.
Companies able to gauge those benefits achieved an 8% increase in revenue and a 10% reduction in costs on average.
Takeaway 5: It pays to invest in Big Data analysis: Companies benefit from measurable results on all levels, from better strategic decisions to better control of operational processes.
Looking more closely at where companies today are utilizing Big Data analyses, we see use cases in many different areas. In the next blog, I will present a summary of the 1,000+ use cases that survey participants entered.
For further reading, download the BARC Survey “Big Data Use Cases. Getting Real On Data Monetization.”