Technical Articles
SAP S/4HANA Business Partner Duplicate Check
Data Cleansing on customer/verndor master data is one of the presteps in ECC on the way to S/4HANA. To keep data clean and reduce duplicates in Business Partners at SAP S/4HANA, duplicate check in BP transaction will help to reduce duplicates.
This blog post is created based on SAP Note 2833993 – ICM: Steps for activation of duplicate check on SAP HANA – new infrastructure via transaction ICM_RULESET
enriched with screenshotes and some guidance.
This blog post is created based on a SAP S/4HANA 2020 Release. All settings will be relevant for newer releases as well. For lower releases please consult SAP Note 2833993 and referenced SAP Notes.
Introduction
This duplicate check is using fuzzy search ICM (Information Consistency Management) based on address data index.
With SAP_BASIS release 755 a new infrastructure for ICM duplicate check based on HANA is provided. With this infrastructure a duplicate check without 3rd party systems can be implemented. Related SAP Notes will show necessary steps to downport this functionality to lower SAP Netweaver releases.
The connection of a third party provider tool to the BAS interface (Business Address Service) allows address data for business partners to be checked (including the address data for contact persons and employee relationships).
The duplicate check can take place in the SAP GUI dialog for maintaining business partners, or also using BAPIs.
A duplicate check is carried out in the following cases:
- Duplicate check of the central address data
- When a business partner is created with address data
- When a business partner is changed
- Duplicate check of the address data for relationships
- When a CP relationship (or an employee relationship) is created with a firm’s address
The check is carried out only if an address was assigned. - When a CP relationship or employee relationship is changed
- When a CP relationship (or an employee relationship) is created with a firm’s address
Version: SAP_BASIS 755 or higher
Below SAP_BASIS 755 please consider SAP note
2834389 – ICM: New infrastructure for duplicate check on SAP HANA (implementation note)
SAP HANA prerequisites:
At least SAP HANA 2 Support Package Stack SPS02
If you want to use the rule set for the address-independent duplicate check, the following minimum versions are required: SAP HANA 2 SPS03 Rev. 37.03 ir SAP HANA 2 SPS04 Rev. 43
Implementation overview
- Check prerequisits
- ICM activation
- BAdI activation
- Activate Rulesets
- Clearing Case activation
Check Prerequisits
Check SAP_BASIS release
Check SAP HANA Release
Check ICM Framework
To check if ICM Framework is available check mentioned in ‘PART 4 – test’ of SAP Note 2834389 – ICM: New infrastructure for duplicate check on SAP HANA should be executed.
Transaction ICM_RULESET
At least all shown Ruleset IDs and View Names should be available in status ‘inactive’.
Check availabilitiy of Views
Check in Transaction SE11 if Views
ICM_PARTNER_ORG
ICM_PARTNER_PERS
ICM_PARTNER_CONT
ICM_BP_ADDRINDP
ICM_EMPLOYEE
are active in ABAP dictionary. Create a new SAP GUI session and run Transaction SE11.
Repeat this step for all of the Views mentioned above. After the check you can close the SE11 session.
In Transaction ICM_RULESET check navigation to ‘Global Settings’ and ‘Maintain Display Fields’ if they are working well.
Global Settings
Maintain Display Fields
ICM Activation
Activation Setup
Run Transaction ICM_RULESET and navigate into ‘Global Settings’.
Dependent on available tables for duplicate check you have to set check on relevant tables active and define a threshold value. Following meaning of available tables:
BUT052 – Relationship Addresses (e.g. used at Contact Persons and Relationship dependent addresses which are synchronized into KNVK)
KNA1 – General Data in Customer Master -> not used is SAP S/4HANA (XDxx transactions are not available any more)
KNVK – Customer/Vendor Master Contact Partner -> not used is SAP S/4HANA (XDxx and XKxx transactions are not available any more)
LFA1 – Supplier Master (General Section) -> not used is SAP S/4HANA (XKxx transactions are not available any more)
MOM052 – Workplace Addresses (used at Employee Business User)
T001W – Plants/Branches (Plant addresses)
BUT000 – BP: General data I (Business Partner)
Recommended threshold values are between 75 and 90. This means duplicates are identified based on a threshold. Higher treshold value leads to a more restrictive duplicate determination. For a threshold value of 90 you can say a duplicate is found with of 90 percent of equal data.
Activate Duplicate check.
Push button ‘Application-Side Enablement (SA17)’
Activate Duplicate Check for GUI Transaction BP
Push button ‘Application-Side Enablement (SA24)’
BAdI Activation
For integration of ICM duplicate check a standard BAdI implementation needs to be activated.
Run Transaction SE18 and enter ADDRESS_SEARCH
List Implementations
Deactivate all active implementations first (marked yellow)
Activate Implementation SIC_ADDR_DUPL_CHECK
Activate Rulesets
Run Transaction ICM_RULESET and activate corresponding rulesets
RULESET_ORG for the duplicate check of business partners (organization and group)
Rule set RULESET_PERS for the duplicate check of business partners (persons)
Note: In SA17, the entry “BUT000, field PARTNER” covers both organizations/groups and persons.
ICM_RULESET contains two separate rule sets for these object types. Both must be activated if you want to use the duplicate check for both organizations/groups and for persons.
Rule set RULESET_CONTACT for the duplicate check of business partner contacts (contact persons for organizations or groups)
Rule set RULESET_BP_ADDRINDP:
Both RULESET_ORG and RULESET_PERS always perform an address comparison during the duplicate check.
If you also want to carry out an address-independent duplicate check for the objects “Organization/Group” and “Person”, activate the rule set RULESET_BP_ADDRINDP as well (or instead). If it is active, this rule set applies for all business partner objects.
Rule set RULESET_EMPLOYEE for the duplicate set of employees on the basis of the table MOM052
As a result all Rulesets are activated.
ATTENTION: Ruleset activation is client dependent. You have activate Rulesets in all relevant clients.
Troubleshooting at Ruleset activation
In some cases it might happen that an error at activation occurs.
In this case indices are not generated yet.
Corresponding to SAP Note 2864015 – ICM: Full-text indexes on table ADRC for ICM duplicate check – generation report ICM_DUPL_SET_FULLTEXT_INDICES
You have to run report ICM_DUPL_SET_FULLTEXT_INDICES with option
As a result you can see confirmation screen.
As a result you can see confirmation screen.
Cleansing Case configuration
With ICM duplicate check you have the possibility to create a cleansing case for postprocessing.
Create number range for numbering object COM_CLEAR
Use Transaction SNRO and enter object COM_CLEAR
Test of Duplicate Check
If a duplicate is identified during one of the above-mentioned actions, a dialog box appears with the relevant options for processing the duplicates.
You can use the following functions in this dialog:
- Navigation to the display of a potential duplicate
- Navigation to the relevant duplicate during the creation of a business partner or a relationship
You can choose whether you want to accept the changes made to the business partner that is being maintained. - Creation of a data cleansing case for further maintenance in transaction BUPA_CLEAR
Information how to handle cleansing cases you can find at SAP Online Help
http://saphelp.ucc.ovgu.de/NW750/EN/8f/c28d5377a0ec23e10000000a174cb4/frameset.htm. - Navigation to the display of the relationship for a duplicate relating to contact person or employee data
- Display of the similarity of the duplicate in percent
Display an existing Business Partner and open BP transaction in a second session. Enter pretty similar data into address fields and check the data using <ENTER> or check button.
With this data check duplicate check will be executed and a Pop-Up appears if a duplicate is found.
Starting from this Pop-Up you can navigate to different actions described above.
I hope this blog post was helpful for you. If so, click on “like” or “share”. Please explore the links below for any further clarification.
Helpful links:
- SAP note 3078395 – ICM: Duplicate check does not display any potential duplicates even though they exist.
- See all questions and answers about SAP S/4HANA
- Follow SAP S/4HANAfor more blogs and updates
- Ask a Question about SAP S/4HANA
- Visit your community topic page for SAP S/4HANA
Hi Andi,
Thanks for the excellent blog. When should we enable ICM duplicate check vs address libraries installed in data services and creating indexes
What I understand is ICM duplicate check can be in SAP(ECC or S/4 Hana) system
1. if the Database is HANA
2. can be use to cleanse data during BP creation
3. Not for existing data by executing report to cleanse.
4. only to detect duplicate address not for address validation or standardization.
so being said that data services via DQM will generate indexes and it will perform the following
1. it can work if HANA or any other Database like DB2, SQL etc in ECC system.
2. Can be use to detect duplicates, address standardization and address validation during BP creation.
3. can be use to cleanse existing data via running reports.
Do you see a scenario where we if implement DQM with data services , do we need ICM duplicate check after the database is upgraded from non-Hana database to Hana database.
your guidance required. can you please share your knowledge.
thanks
Geetha.
Hi Geetha,
as far as I know, DQM is for data validation using micro services only and not for duplicate check. I come to this conclusion because having duplicate check in DQM would mean that every business partner date of every customer is stored in the cloud.
Best Regards
Andi
Hi Andi,
Thanks for responding. DQM option can be without cloud also for your information. Cloud has a diferent pricing model and for bigger clients with huge data DMQ installed and address service files purchase makes economical.
I am unable to view this OSS note
SAP Note 2833993 – ICM: Steps for activation of duplicate check on SAP HANA – new infrastructure via transaction ICM_RULESET
I get the error message
Sorry!
SAP Note/KBA 2833993 is being updated.
What can you do now?
Is the oss note being updated by SAP. I can view all other notes. If not I have to raise an oss incident.
thanks
Geetha.
Hello Geetha,
Did you use DQM or ICM?
My requirement is to duplicate check as well as validate address (Which DQM does) . Is there any way we can use both.
Hi Viraj,
Trying to use DQM with HANA SDQ. unfortunately SAP does not provide any documentation for what scenarios to use DQM with HANA SDQ, DQM with Data services, DQM with microservices and ICM duplicate check.
thanks
Geetha.
Thank you Geetha. We are still exploring which one can be implemented. Is there any link where i can see what all configurations i will have to do if we have to go with DQM with HANA SDQ.
Hello Andi,
I did the setting as your instruction but it does not work. The system still allow me to create duplicate BP without any Pop-Up appears.
When I set a break-point in the badi implementation of ADDRESS_SEARCH, the program even not come there. Is there anything I missing?
Thanks,
Hieu
Hello Andi,,
Sorry please ignore my previous message. It seems my mistake during activate the Badi implementation.
Thanks,
Hieu
Hello Andi,
thank you for this very helpful blog.
We have 3 questions:
We created a clearing case, but we could not solve the clearing case.
Will be one of the two BP´s be deleted?
Or will the deletion flag be set?
Thanks
Eva
Hello Eva,
unfortunately "real" FIORI Apps are not supporting this duplicate check (APP F3163 is a fiorirized GUI). Please see SAP Note 2416027. I'm not that familiar with FIORI Apps. May be there is an newer SAP note available.
Best Regards
Andi
Hi Andi,
You mention that the duplicate check via ADDRESS_SEARCH BAdI runs for transaction BP, but also for BAPIs.
Could you please specify, which BAPIs are covered here? Assuming e.g. BAPI_BUPA_CREATE_FROM_DATA
An interesting use case would be to call a read / search API from an external system (e.g. e-commerce), before creating the Business Partner.
So could it be used e.g. via BAPI_BUPA_SEARCH / BAPI_BUPA_SEARCH_2 or similar to check existence based on address and related data?
Thanks and regards
Stefan
Thank you for the post Andi,
Is there any explanation how the match % (score) is calculated? I see in the code quite complex logic and would like to understand what was the design for calculation.
Thank you,
Marek Turczyński
Hi Marek,
I just took the standard delivery. Unfortunately I don't have any experience.
Best Regards
Andi
Thanks for this really helpful "How-To". Do we have to activate this in each system (dev, test, prod) or is there a possibility to transport it, e.g. the activation of the rulesets in transaction ICM_RULESET?
Thanks & best regards
Yvonne
From my point of view this will be handled via transport request(s).
Best Regards
Andi
Dear Andi,
the BAdI activation is transported - but in transaction ICM_RULESET no pop-up comes up if I activate a ruleset...
Regards
Yvonne
Hello Andi,
Two questions on this:
1. Is there any way we can expand the duplicate check. I also want to include Date of Birth in the duplicate check? How can i do that
2. How is match done. For e.g. if i put duplicate name but completely different address, still the duplicate check shows certain data which should not be the case, as i can have same name in different country. Is there any way to tweak that logic?
Thank you in advance !!
Hi!
Excellent blog. I followed the steps and this helped me to activate the duplicate check successfully.
Really appreciate the details that you have provided on the process. Thank you so much.
Great blog! Thanks a lot! In case you have the same issues as I have and there is no result: have a look at note https://launchpad.support.sap.com/#/notes/3078395.
There was quite a programming mistake:
LOOP AT it_search_results_all INTO DATA(ls_search_result_all).
CLEAR: ls_search_result_all, ls_ex_search_result.
...
Hi Klaus,
thank you for the hint. I added this important note to the blog post.