Skip to Content
Author's profile photo Ido Shamgar

From Big to Simple: how to ensure that Big Data delivers on its promise

For many companies, leveraging the power of Big Data is an imperative for their success.

The ability for organizations to acquire, process and analyze high volume of structured and unstructured data – like transactional data, social media feeds, sensors or geospatial data – bears the potential for them to create new business models and re-define how their industry operate.

But what are some of the challenges and opportunities that stand in front of deciders as they are considering Big Data for their organizations? Here are five challenges you need to get right.


1 – Complexity

Building a Big Data infrastructure and garnering useful insights from it can be a complex endeavor. If not done well, companies can easily be trapped into adding confusion to their decision making with no real value-add. In my previous blog, we discussed a simple strategy to start with Big Data. The key is to start small and focus on a few use cases, get the buy-in from your constituents and measure the impact to your bottom line.

From a technical point of view, IT landscape consolidation will be critical so you can process all types of data from a single platform. You can also use Big Data directly in the cloud, to simplify your environment and avoid major upfront investments.

2 – Data overwhelm

With Big Data, organizations will get access to much more data than they ever had before. One of the main challenges will be the integration of all the data sources into one, and the ability to find meaningful patterns that bring additional insights to your business.

Many decisions can be automated. For example, an online retailer generates targeted promotions automatically based on CRM and customers’ Facebook feeds.

For the most part though, human beings will be at the end of the chain responsible to quickly comprehend the insights and make the best decisions possible. To avoid data overwhelm, it is critical to integrate Big Data into your company’s business processes and identify at the outset how the decisions will be impacted.

Among other elements, you also need to ensure that you have the right tools to visualize the data, so the important information becomes easy to spot and execute on.

3 – Lack of skills

Whether you are leveraging the cloud or not, your Big Data environment will be comprised of many new advanced technologies that your teams will need to master. You may want to use technologies like Hadoop and associated tools to leverage a distributed high data volume infrastructure.

Maybe you want to combine it with a real-time platform to consolidate your transactional and analytical environment and get instant access to critical insights.

You may also need a team of Data scientists with advanced analysis skills to be able to identify interesting patterns and predict future trends.

The bottom line is: Big Data skills are different than your standard Business Warehouse ones. Before getting into the weeds of a Big Data implementation, make sure to have the right end-to-end knowledge in your team.

4 – Executive commitment

Whatever industry you are working in, technology is increasingly at the heart of all the decisions you need to make. Big Data should not be treated as another IT project, but should be at the core of your business strategy. Think about companies like Ford motors, Wal-Mart or Burberry. They all changed their business models to be technology-first, and as such, are building their future success.

Many companies have already hired a Chief Technology Officer sitting in their board, who understands how the industry is being re-shaped through technological innovation. If you don’t have a CTO in place you should at least ensure strong board and executive commitment to make your Big Data project a success.

5 – Change of culture

Connected devices, machine-to-machine communication, business networks or social media are not anymore conceptual trends companies need to account for in the future. They are grounded in reality, disrupting businesses and changing the way we all live, work and interact. What has been dubbed as Big Data is nothing else than the processing of all the information generated by people and devices into meaningful business insights.

More than ever a change of culture is needed, to ensure that companies are ready to tackle the challenges ahead, with the help of what many call the new gold: data.

This article appears also on SAP Business Innovation.

SAP Services can help you define and execute your Big Data strategy throughout your journey. To get more information please visit us on:

Assigned Tags

      Be the first to leave a comment
      You must be Logged on to comment or reply to a post.