Introduction
The SAP Datasphere Intelligent Lookup Series is intended to provide you with useful guidance on how to utilize the Intelligent Lookup to leverage the potential of your data landscape. In order to design business processes efficiently and satisfy customers, a very high master data quality is essential. Often, this is not the case in-house. Fortunately, there are external providers who make optimized master data (e.g. organizational data and address data) available. For successful use of these, they have to be linked with internal data. SAP Datasphere provides Intelligent Lookup, a solution to match external data with internal data that semantically fits together even if there is no common key between these data.
This article is the second in the blog post series and will give an overview about the sorting and filtering options when using Intelligent Lookup.
- Harmonize your internal datasets with external data
- How filtering and sorting enhances your work with Intelligent Lookup (this blog)
- How to integrate an Intelligent Lookup in your existing data landscape
- What is a fuzzy match and why should I care?
- SAP Datasphere Intelligent Lookup Series – Up for a (Data) Challenge?
Please note: All data shown in this blogpost is sample data.
Why should I care about filtering or sorting?
When creating an Intelligent Lookup, we often define rules to match our data. When applying a rule, we usually have unmatched records left. However, not everything needs to be a rule. We can also manually browse the unmatched records by filtering and sorting to match them manually. Especially when dealing with huge datasets the filter and sorting options of the SAP Datasphere can be helpful.
How does it work?
We recommend first matching your records by applying rules and afterwards going through the unmatched data to match it manually. You can sort any column ascending or descending or even already filter by clicking on the column name.
Figure 1: Sort records by column (Source: Own image)
Sorting can be especially helpful to get more familiar with your data and understand why it was not matched, yet. In our example we use sample sales data from a web shop that is supposed to be enriched by geo data from a Google BigQuery dataset. Both was already used in this blogpost: https://blogs.sap.com/2021/12/22/intelligent-lookup-video-end2end-walk-through-and-diy-exercise/
After applying the first two rules there are still 2873 unmatched records. When sorting by city_name we can see that a lot of records start with an apostrophe or a number and some are not city names at all:
Figure 2: Sorted overview for city names (Source: Own image)
Finding a match through sorting, but without applying any filters might not lead to success. You can either apply one or even more than one filter.
In this example we will apply two filters:
Figure 3: Unmatched rows with multiple matches (Source: Own image)
Figure 3 shows that we have 1623 match candidates in the multiple tab. When we look at Frankfurt there are multiple records that could be potential matches. To find the correct one we can either use the sorting option and sort by score or we use the filter option. You can find the score in the first column of the GlobalCities table. This suggests how well the data matches your selected data on the left. For filtering you can follow it step-by-step:
- Click on the settings-symbol on the top right of the table.
- The View-settings open. Please note: The filter can only be applied to columns that are selected here. Click on the “Filter” Tab.
- Type in your filter rule. The city we want to match is known as “Frankfurt” but officially named “Frankfurt am Main”. In our case we want to include all the cities in Germany that contain the name “Frankfurt” and exclude the region “Brandenburg” as we know that there is also a city named “Frankfurt an der Oder”.
- Click on “ok”.
- Check the result and match the record or go back and adjust your filter. The applied filter is highlighted in green, and you can delete the filter by clicking on “Clear Filter” on the top right of the table.
Conclusion
In this blog post, you learned about the filtering and sorting options when using an Intelligent Lookup.
The filtering and sorting option is especially helpful when…
- …you want to get to know your data
- …you have already restricted your data and have records left that cannot be matched by a rule
- …you are dealing with huge datasets
Thank you for visiting this blog post. I hope you found it helpful. Feel free to use the comment section below for any feedback or further questions. If you want to learn more about Intelligent Lookup, I recommend you go through the other posts in this series. A big thank you goes to my colleagues Tim, Florian, Josef and Richard for collaborating on the blogpost series.
Best wishes,
Carolin
Further Links
- One-Stop-Shop to Intelligent Lookup in SAP Datasphere
- End2End Walk Through – Intelligent Lookup
- Intelligent Lookup Documentation in SAP Help Portal
- A deeper look into how SAP Datasphere Enables a Business Data Fabric
Find more information and related blog posts on the the topic page for SAP Datasphere
- SAP Managed Tags:
