CRM – Why is it so important to take care for Data Quality?

This article briefly describes the 3 cornerstones to ensure high Data Quality in CRM: Address Validation, Duplicate Check and Data Cleansing. It also describes options how this can be done in a more automated process.

In the past I’ve posted already some content for this community about merging redundant Account Records in different Scenarios (the related Links are listed at the end of this Blog).

Well, to take care for Data Quality in CRM is a very important topic and I like to add some of my thoughts here again …

Some day’s ago I’ve seen the famous movie “Matrix Revolutions”. I guess most of you have seen this already. In “Matrix Reloaded” Agent Smith has started to create duplicate records of himself and in the next episode (“Matrix Revolutions”) this was not only a threat for our heroes Neo, Morpheus and Trinity, but for the whole system.

In CRM that’s very similar. If data quality is not ensured not only single users are facing serious problems – at the end the whole system will not longer work efficiently.

The Sales Rep. is not able to identify the right Accounts he should focus on. A Marketing Campaign is very expensive and can’t reach the right potential customers. Analytics will work with wrong data and this will cause wrong conclusions and wrong decisions.

For sure it’s easier to get rid of duplicate Account Records in CRM as to get rid of Agent Smith in “Matrix Revolutions”. However it’s easier to avoid duplicates and I should not longer talk about Science Fiction …

This whole topic is very important especially in CRM as this is the main area where new Account-Records gets entered even if only some parts of the address- and communication data are known. Data Quality in CRM is very critical as this very often is a neglected topic. If a server crashes or if any functionality doesn’t work a company immediately takes the right action. Bad Data Quality is rather acreeping disease.

The first important aspect is that every user is aware about his responsibility for good data. It’s not only him, who is working with the data and not sufficient, if he just maintains a phone-number because everything else is not important for his current tasks.

In this case it is very helpful to have a clear assignment of responsibilities. However this awareness will never be sufficient and usually it fails if users are just forced to enter accurate data.

It’s something every customer has to take care for within his CRM-Solution to also implement solutions supporting the user in entering correct data, to prevent the entry of duplicate records and to remove redundant records.

Online Address Validation

Here a first important option to support the user is to integrate a powerful Address-Validation which automatically validates the address based on referential data sources, and format the address according to the norms of the applicable country.

So if the user enters an Address the system will automatically suggest a corrected and completed Address the user can select. Dependent on the logic of the Solution for Address Validation some other address-related information like Tax Jurisdiction Code or Geocoding-Data can get added. In addition some information can get driven out of the combination of Address and Name to automatically update fields. E. g. the First Name “Andrea” could indicate the gender of the person but this is dependent on the country. In Germany “Andrea” is most likely a name for a female person but in Italy it is rather a masculine name.

In Korea “Kim” is rather a last name, but in other countries it is rather the first name.

There are several solutions available at the market which can get integrated for this purpose. In principle all these solutions are based on the Business Address Services Interface (BC-BAS-PV for Business Address Service Postal Validation).
Several 3^rd Party solutions are certified on this Interface. You can find several 3^rd-Party-Solutions within the Partner Information Center if you search for Solutions based on BC-BAS-PV (Business Address Service Postal Validation).http://www.sap.com/partners/directories/SearchSolution.epx

Here you can also integrate “BusinessObjects™ Data Quality Management for SAP^® Solutions”which works for SAP ERP and SAP CRM.

Further information about “BusinessObjects™ Data Quality Management for SAP^® Solutions” you can find here:

http://www.sap.com/solutions/sapbusinessobjects/large/information-management/data-quality-management/dqm/dataqualityforsap/index.epx

Online Duplicate Check

Based on complete and accurate Address Data the next level to ensure good data quality is the online Duplicate Check. The Online Duplicate Check works whenever a new Account gets created or if relevant fields get edited. Whenever there are similar Account-Records (above a defined threshold of similarity) the user can check, if the current record already exists in the system and if he wants to continue with an existing record, still create a new record or if he wants to merge duplicate records.

In CRM you can use here the lean TREX-and-BAS-based duplicate prevention functionality (BC-EIM-IQM-IC) which is part of SAP NetWeaver. However it is also possible to use more sophisticated 3^rd-Party-solutions. Here again you can find some 3^rd-Party solution within the “Partner Information Center” if you search for Solutions based on BC-BAS-DES (Business Address Services – Duplicate Check)http://www.sap.com/partners/directories/SearchSolution.epx

Sophisticated Duplicate Check capabilities are also integrated in “BusinessObjects™ Data Quality Management for SAP^® Solutions”
(http://www.sap.com/solutions/sapbusinessobjects/large/information-management/data-quality-management/dqm/dataqualityforsap/index.epx)

A lean duplicate check only considers fields of the Address and the Name of the Account. A more intelligent Duplicate Check also has some kind of fuzziness and logic to identify potential duplicates. As an example “Beth Meyer” and “Elizabeth Mayer” could be duplicate records for the same person. “General Electric” and “GE” could be duplicate organizations.

In addition other attributes not belonging to the Address are also relevant to identify duplicates (e. g. Birth date, Identification-Numbers, e-Mail, Phone-Number etc.).

Data Cleansing

Usually the Duplicate Check can prevent the creation of many duplicate Account Records … but not all. Duplicates are always in the system and usually the number is growing. Therefore it is required to create Cleansing Cases in order to merge redundant Account Records. This functionality is available in CRM and is in principle also independent from the Duplicate Check as such. The option to create Cleansing Cases is available for all Account Search-Result-Lists and in addition (if a Duplicate Check has been activated) on the Duplicate Check Popup.

The creation and the processing of Cleansing Cases are decoupled. This means the User can create a Cleansing Case and he merges the Accounts directly or he or another User merges the Accounts later.

At the end there should be one remaining consolidated and enriched Account Record and all Non-Master Accounts are flagged for archiving.

Data Quality Administration

For CRM users (especially for Sales Professionals), time has never been more precious as today. Therefore there has to be also an option to automate and to outsource tasks around Data Quality. The CRM Data Quality Administration Framework is one solution for this as it offers options to automatically de-duplicate Target-Groups including thousands of Accounts. It also offers the option to export Data-Files and to import validated and enriched Data from an external agency again.

Additional Information

Overall:

• Blog: CRM 2007: Different use-cases for merging duplicate Account Records
  http://www.sdn.sap.com/irj/scn/weblogs;jsessionid=(J2EE3417600)ID1322377650DB01061339403520123576End?blog=/pub/wlg/1097
• Note 176559 – Interfaces for Business Address Services
• SAP Help: Duplicate Check and Error-Tolerant Search
  http://help.sap.com/saphelp_nw04/helpdata/en/a3/eaa43ab9db4814e10000000a11402f/frameset.htm
• Note 1001468 – Duplicate Prevention will not be further developed
  https://service.sap.com/sap/support/notes/1001468

… related to “BusinessObjects™

• Data Quality Management for SAP^® Solutions”
  http://www.sap.com/solutions/sapbusinessobjects/large/information-management/data-quality-management/dqm/brochures/index.epx

…  related to Data Cleansing:

• Note 1090391 – FAQ : Business Partner Data Cleansing
• Video about Merging Accounts in CRM WebUI:
  https://www.sdn.sap.com/irj/scn/elearn?rid=/library/uuid/1081a104-fd62-2c10-74a9-d36c6ecc3cee

… related to Data Quality Management:

• Consulting Note Note 1140790 – Data Quality Administration: Customizing & Enhancements
• eLearning Video (Data Quality Management Task: Integrated Duplicate Check)
  https://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/40bcfd1a-4598-2b10-f590-cb808aa6bc53