Data Quality Audit

[amitharayil]

One of the indispensables, essential and dominant parts of our lives is our smart phone. You can almost say there is nothing a smart phone cannot do nowadays. However, technological advancements do not stop there, we have continuous innovations been introduced into our surroundings.

Over 20 years, we saw the first mobile phones with antennas that evolved into faster, lighter, graphical, and more intuitive devices. This is a splendid example of technological transformation.

We started off with just voice communication over analog signals which transformed into so much more. We can now access the world’s news and information at our fingertips, communicate and network within our circle - even with strangers based on our geographic location - shop, trade, manage finances and organize our lives globally from wherever we may be.

As mobiles get smarter and smarter, think about the data that has been transferred from one device to another. And these transfers came with several implications and adjustments to the data over the years. For instance,

• Compliance wise – with the onset of international calling, phone numbers were suddenly required to be stored in international format.
• Competition wise – amongst varied brands, more features kept pouring in terms of speed, additional fields, and memory to store more contacts and addresses.
• Looking at technology trends – devices now have the power of internet telephony, connecting themselves to self-driving cars and comprehensive mobile applications to become a pocket computer.

Data quality is the least of our concern as we are focused on the outcome to move to the new device and get started, by just ensuring that we retain the data in the new device in some form. The advanced intelligent machines assume implicitly that the data transferred to them is suitable for their intended purposes. We all know that transfers across platforms and vendors are limited by data models and technical limitations.

Consider this scenario: during the transfer of data between different device platforms, envision a situation where contact numbers or addresses are jumbled. You may have been consuming this data over years across devices manually, knowing the limitations and issues in the data, and deciding accordingly e.g., Mobile phone and Office phone fields have data interchanged during transfer, and postcode missed in the addresses during transfer from Apple to Google platform. Now, imagine relying on this data on your mobile device with a self-driving car to navigate you to your destination e.g., AI considering only street name to navigate and deciding between multiple options!

How can you trust the data that has been moved from your old devices into the newest device via multiple devices and platforms?

Would you be able to use the full capability of the new AI self-driving cars?

Is the data that we have been maintaining for decades fit to be used in these innovations without human intervention?

Or more simply, is the data fit for the new intended purpose?

Now coming back to the corporate world and considering the business point of view for customers who have moved their enterprise data assets across multiple platforms over last few decades to SAP ECC or any other ERP system. Most of these customers are now in the process of transforming their technical landscape and business processes to get ready for the future, i.e., Cloud! Whether these organizations are moving to private or public cloud, RISE with SAP or GROW with SAP, going greenfield or brownfield; the transition mirrors the evolution we discussed earlier with mobile phones but, in this case, businesses are shifting from their familiar ERP ECC systems to the futuristic landscape of SAP S/4HANA. It involves migrating from mixed landscapes, including SAP, non-SAP, and legacy systems, into the innovative SAP S/4HANA platform. The AI and Generative AI capabilities embedded in SAP S/4HANA mimics driverless cars we mentioned before. So, we find ourselves asking the same questions once again:

Does the data make sense for the S/4HANA system? Can we take advantage of the GenAI capabilities?

Can we rely on and trust the data for the system to predict your business outcomes or take business decisions?

In the end, is the data fit for intended purpose?

Data quality (DQ) is always the key focus and concern from the business at the beginning of the transformation, however as program evolves, DQ topic fades as the business priority moves to ensure processes are designed and tested. The challenge is that no one could really quantify and establish the problem to define a business case to invest in data cleansing to save costs for the organization. By the time business users realize the scope of issues through migration, it is too late to fix this problem and they are already testing the processes. If at the end they are successful in migration, they still end up with technically correct and not “business correct” data set.

This helps us see those businesses, like individuals with mobile phones, need to make sure their data is ready for the changes and innovations in their systems.

How do we ensure data is fit for purpose?

Let us address this topic again with the same mobile phone analogy. Consider this scenario: you are moving from an old Nokia phone to an iPhone. The former mobile phone saved contacts separately and did not have any tagging features to link landline or mobiles numbers of the same person together. However, the new iPhone has the feature to add multiple numbers with tags to a single contact. Now when you are moving to the new phone quite literally, the contacts can become redundant in your new device and cause confusion. How would you be able to tell apart which number is landline and which is mobile?

Similarly, what if some mobile numbers were transferred incorrectly and lead to missing information in your contacts? Or during transfers and conversion of formats to comply with international codes, incorrect patterns were transferred, or the numbers truncated?

This would affect your new device’s ability to function as expected and for you to utilise innovative features on your contacts.

Many such examples can come up in various parts of the transfer journey between the devices especially when you are moving between technically diverse landscapes. How do we ensure we do not move junk or bad data into our systems? Or if we already have moved without auditing the data first, how can we identify inferior quality data in our current landscapes?

The first step to check if data is fit for purpose is to understand the depth and extent of your data issues. To understand your data issues, you need to audit the data. The data may appear clean, but the question is if it’s fit for the intended purpose.

In the business world, the information spread across your systems is undeniably the most valuable asset. Achieving 100% data quality is a challenging task for any organization, but depending on how crucial each aspect is to the business, you can prioritize and make it feasible over time. For example, if there are 100 fields or attributes that require attention, and only 70% of them are essential for your business operations, you can identify key areas to concentrate on and begin planning for improvement. So, where do we begin?

What is Data Quality Audit?

Data Quality Audit, a service offered by SAP, assesses, and checks the actual data stored in systems to uncover the existing issues and determine fitness for target systems like S/4HANA, C4C, ARIBA, MDG etc. Irrespective of the number of systems you have, or at which stage you are in your transformation journey, Data Quality Audit checks the ‘quality’ of the data in terms of ideal principles set by your business and generates a ‘scorecard’ along with a business impact ‘cost analysis’. This allows organizations to assess the status of their data and then help formulate their cleansing, resource and migration plans accordingly.

Let us briefly look at the steps that this service follows to achieve the results:

Once scoping is established, data quality ‘rules’ are defined on the data with business or IT. The ‘rules’ originate from ideal principles of data quality: Accuracy, Completeness, Conformity, Consistency, Integrity, Timeliness, Uniqueness. SAP offers best practices content of data quality rules as a QuickStart to any quality audit.

These ‘rules’ are then executed on the data, and ‘scorecards’ are established in a dashboard-like view. The scorecards also depict the associated cost impact analysis for the poor data quality. This then helps enterprises to assess their data, constantly review their data during the cleansing activities, supporting control and governance.

To conclude,

Data Quality Audit is a service that is pivotal in the initial 'Get Clean' phase of your transformation. Unfortunately, these aspects are often underestimated, leading to the failure or delay of numerous transformations. Taking these steps seriously is crucial, as neglecting them can jeopardize the success of your large-scale investment, impede growth, and hinder the identification of business issues.

Consider these three key takeaways:

• Recognize the vital role of Reference Data Quality Audit and Data Quality Audit in achieving a clean transformation. Look out for a similar blog on Reference Data Quality Audit - stay tuned!
• Be mindful of the commonly underestimated nature of these steps, as it can significantly impact the success of your initiative.
• Formulate robust 'Stay Clean' and ‘Get Clean’ plans based on the insights gained, avoiding pitfalls that have derailed many transformations."