MDM and Big Data – Discovering the obscure link

In the following blog I have tried to detail my understanding of the subject.For a clearer understanding I have kept it in a question answer format.Please feel free to add to it.Here we start.

Big data and MDM seem to have a significant connection, but for now the connection is still very unclear. At first thought they seem like an odd pair with great contrast between them.

MDM can be defined as a set of tools, procedures, and policies to govern, create and maintain a trusted data. This data serves as the very base of business transactions and hence many times referred to as “DNA of a business”.

Big Data on the other hand comprises of environments which is composed of huge volume of data coming from variety of sources. It is the overall umbrella of social media data, unstructured documents, streaming data from instrumented devices, and more. Unlike MDM its main character is not Trust.

In Gartner’s words “Big data is high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making.”

So how are the two related?

Big data provides a high volume data, this high volume and high velocity data needs to be processed and refined to come at valuable information. And here lies the link – The MDM hub can keep a traditional ‘golden record’ of trusted information along with a less-trusted view of the same person or product based on the findings from big data. When you combine the traditional ‘golden record’ with new information found from  your big data, the superset of information can power even better business insights and business decisions that were not possible before. The combined view can provide a more insightful complete view, but they can still be presented separately in cases where business can’t afford to base decisions on the less-trusted view. So Big data is supported by MDM, it tells a business what does a predictive analysis or classification mean for its Customers, Vendors and more.

How to arrive at valuable information?

To achieve this business would have to mine the Big data. In other words, it would involve exploring and analyzing large amounts of data to find patterns for big data. The techniques came out of the fields of statistics and artificial intelligence (AI), with a bit of database management thrown into the mix. Generally, the goal of the data mining is either classification or prediction. In classification, the idea is to sort data into groups. For example, a marketer might be interested in the characteristics of those who responded versus who didn’t respond to a promotion. Based on such information he can change his strategy to target a group of responders. To add to this there are softwares enabling a process called Opinion mining. It is a type of natural language processing for tracking the mood of the public about a particular product. It is also called sentiment analysis, involves building a system to collect and examine opinions about the product made in blog posts, comments, reviews or tweets. Automated opinion mining often uses machine learning, a component of artificial intelligence (AI).

Together big data and MDM can help extract insight from the increasing volume, velocity, variety, and mapping that to the truthful data, in context, beyond what was previously possible. This could lead to creating master data entities, loading new profile information into the MDM system, sharing master data records or entities with the big data platform as the basis for big data analysis, and much more.

Example –Let us take example of Health care system. Here master data could be Patients, Providers, Payers, Households, Employees, Reference data etc. To have complete view of a Patient we can have data from Hospital, Doctor, Web, Health insurance, Pharmacy etc. One can leverage predictive analysis to come at patients who are at risk for readmission, leverage classification to measure and identify causes for adverse events, assist in drug discovery ,this can help insurance companies devise personalized programs etc.

How do innovations in existing MDM portfolio enable this connect?

SAP is employing a multipronged approach.  Firstly, the current Master data solutions would continue to evolve in line with business, addressing their needs. Secondly, SAP is leveraging SAP HANA as the common data platform for SAP enterprise MDM .It would allow businesses to handle huge volume of data in large scale master data hub.  Additionally, the Master Data Services package from SAP and Business Objects enables customers to improve business process efficiency, effectiveness, and responsiveness, and make better informed decisions based on high-quality, accurate master data.

I would like to conclude by saying that to have MDM in place is to already have a strong foundation for Big data,together they can provide a Vantage point to business.