How Machine Learning Accelerates Data Management

Welcome to the third post in our Data Intelligence introduction series. You can find part 2 here.

Imagine that you own a road that leads to a small village in the middle of nowhere. You can easily manage the traffic on your road, until a huge natural resource in this village is discovered by accident.

A major resource company comes in, bringing hundreds of vehicles along with it. Public transport options increase. Massive trucks begin using your road on a regular basis.

But your road was never designed for usage like this. You have two choices. Would you invest in new infrastructure to expand and improve the road and perhaps add more roads? Or would you restrict vehicles to keep traffic at an acceptable level?

Many companies find themselves in similar situations when it comes to data.

The Challenge

Traditional IT enterprise architecture is designed for taking care of structured enterprise data. But as time goes on, companies amass a huge amount of unstructured data, such as audio files, video streams, and social media impressions.

This data usually resides in object stores and data lakes scattered across company’s IT landscapes, where it’s a challenge just to find it. Traditional enterprise management software is unable to keep up.

As we mentioned in our last article, most companies are convinced that applying intelligent technologies like machine learning to enterprise and unstructured data will help them gain smart, valuable insights to assist with business decisions. When it comes down to it, though, replicating an entire data lake into an application or even a system usually isn’t feasible, for a variety of reasons.

This creates a storm of questions: how do we leverage and automate unstructured data into machine learning scenarios? How can we more efficiently find and identify that data? And how do we make all of this sustainable?

The answer

We built SAP Data Intelligence to tackle these challenges.

SAP Data Intelligence enables users to handle their enterprise data through metadata governance and data quality processing, along with discovery, lineage, preparation and transformation. Data Intelligence Cloud then helps users integrate and orchestrate their data through complex data pipelines with comprehensive support when it comes to operationalized intelligent scenarios.

But that’s not all SAP Data Intelligence can do. The key concept behind an intelligent enterprise data management solution is leveraging the efficient use of extracted metadata assets.

With SAP Data Intelligence, metadata assets from related source systems can be made available in the Metadata Explorer catalog for labeling, searching, and data lineage functionalities.

Last but not least, there’s another level of enterprise data management consisting of the symbiosis of classical data management functionalities and intelligent technologies like machine learning. For example, one could use machine learning techniques to:

Automatically classify extracted metadata assets based on text analysis, image processing, or natural language processing.
Automate the mapping of predefined entities (like sales orders or customer industries) as well as establish and maintain the relationships of extracted entities originating from various connected systems.

The establishment of a so-called “self-learning metadata governance” is stated on the roadmap of SAP Data Intelligence.

This functionality boils down to the combination of metadata management and machine learning enriched with business semantics, which are extracted from SAP business systems and applications. It represents the metadata management of the future and will enhance the productivity of involved subject matter experts.

Data Intelligence Cloud is just getting started. Want to stay in the loop? Subscribe to the Data Intelligence Cloud newsletter below!