Skip to Content

Finding duplicate records with SAP BusinessObjects Postalsoft DeskTop Mailer and Business Edition

The most common questions we hear in support are “Can the software find duplicates”, “How do I get started”, “What rulebook do I use” and “Why didn’t the software find duplicates”. The information in this blog is very basic, just to get you started.

The first step is deciding what you are targeting with your mailing (assuming you are mailing).

  • Are you going to mail one piece to each address regardless of who lives there
  • Are you trying to mail one piece per family at the address?
  • Are you mailing one piece per person at the address?
  • Are you targeting businesses; one per company or to each person working there?

Once this has been decided, look at how you have your Postalsoft file layout set up (File > Properties > Database). Which fields contain the data and what are the field kinds? I’m sure you are asking, what do you mean field “kinds”? Postalsoft recognizes the field kind when processing your list whether you are running Address Correction, Merge/Purge, or presorting (it does not look at the name of the field). The “kind” needs to be unique and one that is recognized by the software. So, let’s say you want to import a new list and you want to mail one piece per family at a given address. This means you want the software to look at the 5-digit ZIP Code, street address, and last name. That way, no matter how many people are at that address with the same last name, only one person will receive the mailing.

The Merge/Purge in Postalsoft has several rulebooks predesigned for you to choose from. A rulebook is designed to instruct the software which fields you want to compare, which fields are the most important in the comparison, and how “strict” you want the comparison to be. Meaning, do you want the data to be exact or allow for minor spelling differences? Several predefined rulebooks are installed with the software: Resident (1 piece per address), Family (1 piece per family at an address), Individual (1 piece per person at an address), Company (1 piece per company at an address), and Company Individual (1 piece per person at a company at the address). These rulebooks are also designed to be Exact, Tight, Medium, or Loose it all depends on what you decide to use. The “strength” of the rulebook determines how exact the data needs to be as a match. For example, I want to mail a piece to each family in my list, but I want to be sure the street address and last name are 100% alike or I don’t want them to be found as duplicates, so I’d need to use a Family rulebook. You can customize one of these rulebooks or make your own.

To get started you must first import your list and run Address Correction (CASS) or send the job for NCOALink via Extended Services which also performs the CASS if these steps have not been done prior. This will standardize the address data to make it easier to compare. Either process will move your primary “assigned” address to the Street Address line 2 field. Now look at your job again after processing to verify what fields the data is in, and pick your rulebook during Merge/Purge. In the below example I’m going target one piece of mail per family at each address. Since the last name data is in the Last Name 1 field, and address data that was assigned is in the Street Address line 2 field, I will choose an Address 2 Family Exact rulebook.

  1. Click Find Duplicates on the Task pane, or click Tools > Merge/Purge
  2. Only the one file you have open should be listed, and it should be listed as an “Input normal list”. Click Next.
  3. Under Compare records choose a prequalifier. What is a prequalifier? The prequalifier is used to reduce the number of comparisons to make during the duplicate detection, which can speed up the entire process. Leaving this on “Same 5-digit ZIP” is the most common. Whichever field you choose here, has to be indexed under File > Properties > Database. The software will now group records by 5-digit ZIP and compare records only if the 5-digit ZIP matches. For instance, a record with a 54601 ZIP Code will not be compared against a record with a 54602 ZIP.
  4. Choose a rulebook after deciding which fields to compare. I determined above what I want to compare so I’ll choose the “Addr_2 Family Exact.mpr” rulebook. The software will compare the 5-digit ZIP, Last name 1, and Street Address line 2. Click Next.
  5. Under Input file(s), choose the action to perform. The program default is set to “Delete all but highest priority dupe in every dupe group”. With this choice, the first record (one with the lowest record number) found in the dupe group will be kept as the highest priority dupe.
  6. Under Report generation, select the reports to be generated if desired (this is optional).
  7. Under Interactive control, I would recommend choosing “Show summary of dupes found with opportunity to change”. With this option, the software will show the dupes found so that you can decide if the rulebook you chose worked like you wanted, and also decide if you want to make changes to which records that get deleted (the ones with check marks get flagged for deletion). Click Next.
  8. On the summary screen, review the options chosen (click Back to change any options).  Click Finish to start the process.
  9. If no dupes were found the software will produce a message stating this, otherwise the “Review Duplicates” screen will be presented. Review the duplicates found and if everything looks the way you expected, click Perform Actions.

The records were flagged for deletion because of the choice I made in step 5; they are not permanently removed at this time. To permanently remove them, click Tools > Remove Deleted Records.

Easy right?? That is it for a simple Merge/Purge run. If you know what you are targeting with the mailing, and what fields you want to compare that is half the battle.

If you have more questions the SAP Knowledge Base has a large amount of KB Articles to assist you, the Postalsoft Help Topics or User’s Guide are also very good tools. SAP BusinessObjects User Support is always willing to assist you as well!

Refer to the Postalsoft Wiki for How To’s and FAQ’s as well.

Be the first to leave a comment
You must be Logged on to comment or reply to a post.