Welcome to the first post in our Data Intelligence introduction series. In this article, we'll be talking about how machine learning and information management work together to build insight.
What does your company need more: Machine Learning (ML) initiatives, or better information management?
For any forward-thinking company, the answer should be both. But it’s often a question of prioritization, and many are choosing to focus on information management.
According to a Forrester survey of 178 global IT decision makers, 82% of organizations believe that orchestration and automation of data processes and workflows is crucial to their success. However, only 37% of the same respondents were confident in their ability to successfully accomplish this.
At the same time, 67% believed implementing ML processing was critical, but only 33% felt confident in their ability to do so. The challenges are inextricably linked, as data feeds ML, and ML requires information management to be effective.
Unlocking the Promise of AI
If you were to believe the buzz, AI could automatically solve our biggest problems with complex computer wizardry, granting all of us a life of leisure and simplicity.
AI’s potential has been touched on in countless works of science fiction. But this reminds me of two in particular.
First, we have WALL-E, Pixar’s animated movie about a lonely robot cleaning up garbage on earth. Humanity has left the planet for an AI-driven spaceship, and our every whim is taken care of, leaving us practically immobile. We move around in futuristic scooters, which we don’t even have to drive, and have no idea how to live for ourselves.
The second is The Hitchhiker’s Guide to the Galaxy (originally a book, but there’s a movie too), in which hyper-intelligent beings design a computer to reveal the answer to the meaning of life, the universe, and everything.
After processing the question for 7.5 million years, the computer announces that the answer is “forty-two.” And its creators realize that they never knew what the original question was anyway.
In both, the problem was information management. The people of WALL-E had their information strictly controlled by AI. And the beings in The Hitchhiker’s Guide to the Galaxy didn’t even know they were missing information until they had their answer.
Despite being sci-fi, these two stories parallel modern times: even now, information management approaches have failed to keep up with technological change. Most data technology was built and designed for the days of on-premise applications interacting with on-premise databases.
Here, the goal is to extract data and load it into a data warehouse for business intelligence and reporting. While that need still exists, the data that we manage and the ways we extract value from that data have radically shifted and diversified.
Today, we face a complex mix of structured, unstructured, and object store data residing in a blend of cloud and on-premise systems, with access often being limited or non-standardized via APIs. The result is a complicated landscape of data sprawl, tooling diversification, and data siloes.
All of this makes it harder than ever to “locate the wisdom we have lost in knowledge” and the “knowledge we have lost in information,” to quote T. S. Elliott.
Where Traditional ML and Information Management Fail
ML and information management can do better together. The following data points highlight the need for improvement:
- 86% of enterprises say they should be able to get much more out of their data
- 5 out of 10 early data science initiatives fail to get to production
- 74% say their data landscape is so complex that it limits agility
And perhaps, the most telling: while two-thirds of businesses consider machine learning and AI important initiatives, only one-third or less are confident in their ability to implement such initiatives.
A clear majority of companies recognize the importance, and potential benefits, of these initiatives. So how do these businesses build up their confidence in order to go through with them?
Data Orchestration = Data Integration + Data Processing (ML) + Automation
This is why we’ve developed an entirely new solution from the ground up, with open source and cloud principles in mind. Enterprise data when combined with data streams and big data contains great promise: accessing it, unifying it, and using it is no longer out of reach.
When you bring comprehensive access to all data together, with the application of data processing engines like ML, and connect the results to business processes and workflows, you get true data orchestration.
SAP Data Intelligence Cloud was designed to allow any organization to orchestrate and automate data and business processes while also deploying and operationalizing machine learning
For more information on data orchestration and how it helps you extract value from data and connect it with agile and reimagined business processes, check out our next post
here.
