Names and postal addresses are useful starting points when comparing individual customer records. Email addresses, telephone numbers and existing customer identifiers provide additional evidence where those details are reliable. Organisation records often require company names and business identifiers instead.

The value of each field depends on how your database is used. Members of one household sometimes share contact details, while a customer who has moved appears under different addresses. A shared phone number or postcode therefore provides limited evidence without other supporting information.

In data deduplication, reliable customer identifiers deserve greater weight than common names or shared addresses. Where records lack important details, matching rules should flag uncertainty rather than treat missing information as agreement. This helps your team concentrate on records where the available evidence is incomplete or conflicting.

How do exact and fuzzy matching identify duplicate records?

Exact matching identifies records containing identical values in selected fields. Fuzzy matching recognises similarities where names or addresses have been entered differently. In data deduplication, these settings determine which potential duplicates appear in the results and how much further investigation they require.

Exact matching: Finding identical customer details

Matching customer identifiers or identical combinations of names and contact details often reveal straightforward duplicates, particularly after repeated imports.

Problems arise when those details are entered inconsistently. Robert Smith and R Smith, for example, are different text strings despite potentially referring to the same person. Exact comparisons overlook this variation, along with differences in how addresses are formatted.

Fuzzy matching: Recognising name and address variations

Standardisation brings customer information into more consistent formats, making variations easier to recognise. Fuzzy matching helps identify records containing spelling differences, abbreviations and similar details. Our Duplicare software also supports fuzzy person name matching, helping identify potential duplicates where names contain spelling differences or recognised variations.

Our Deduplicate API documentation describes name-and-address matching levels for the Deduplicate cleansing service. These differ from the Dedupe+ matching rules configured within Duplicare for Microsoft Dynamics 365:

• Forename: Requires a fuzzy match on both forename and surname, without accepting a single initial as the forename.
• Initial: Matches R Smith with Robert Smith, while distinguishing full forenames such as Richard and Robert.
• Surname: Supports broader surname-based matching even when forenames differ.
• Personal: Selects an appropriate matching level based on the available name information.
• Premise: Identifies records associated with the same address, subject to applicable company-name checks.

At scale, deduplication depends on selecting matching levels that reflect your records. Broader rules generally identify more potential duplicates, but they also increase the number of unrelated records needing review. Stricter rules reduce that workload at the risk of overlooking records with incomplete or inconsistent details.

At premise level, for instance, an address match does not establish that everyone living at the property is the same customer. The matching criteria therefore need to reflect whether the records represent individuals or shared addresses.

Address variations also feature in our AMI International case study. While refining Duplicare’s matching rules in Microsoft Dynamics 365, AMI International identified previously missed duplicate addresses where the property number appeared as 10 in one entry and Ten in another.

What does a duplicate detection result mean?

A duplicate detection result shows which records meet the selected matching criteria. Your team receives a clearer picture of suspected duplicates and the related entries that need investigation.

The Deduplicate cleansing service documentation describes output fields for identifying matching records in processed data, including Duplicate Flag, Duplicate IDs and Keep Flag. It also assigns each processed record a Unique ID.

• Duplicate Flag: Marks an entry Dup when it is identified as a duplicate at the selected matching level.
• Duplicate IDs: Lists the unique identifiers assigned by Data8 to other matching records, helping teams locate related entries.
• Keep Flag: Uses 1 for a record marked to retain and 0 for one identified as superfluous in the cleansing output.

With Data8 Online, we offer options to flag matches or suppress duplicate entries in the output. In practice, flagged results give your team a way to examine candidate groups before making changes to CRM records.

The Keep Flag serves a different purpose. It determines which entries remain in the configured cleansing output and does not authorise a merge in your live CRM. That decision still depends on whether the records genuinely belong to the same customer.

How do you confirm whether two customer records are duplicates?

A suspected duplicate needs supporting evidence. Names, addresses and other identifying details should point to the same customer, with significant conflicts resolved before merging.

Two people living at the same property sometimes have similar names and shared contact details. Meanwhile, one customer might appear under different addresses following a move. Existing account identifiers, contact information and customer history help distinguish these situations.

Conflicting identifiers or incompatible account histories warrant closer investigation. Where the evidence remains inconclusive, retaining separate records protects against incorrect merges.

Recording accepted and rejected matches helps refine deduplication rules over time. This becomes particularly useful when the same records repeatedly appear as suspected duplicates despite being confirmed as separate customers. Within Duplicare, our exclusion controls help teams manage these known non-duplicates during CRM matching.

What should you check before merging customer records?

Once a duplicate has been confirmed, attention turns to which record should remain and what information needs to be preserved. Incorrect merging risks overwriting current details, disrupting linked CRM records and losing useful customer history.

The main considerations include:

• Master record: A clear reason for selecting the record that remains.
• Conflicting fields: Reliable evidence supporting the most accurate names, addresses and contact details.
• Related information: Continued access to transactions, activities, notes, cases and associated records.
• Communication preferences: Appropriate preservation and resolution of consent information and contact preferences under your organisation’s policies.
• Approval and recovery: Tested merge rules, suitable backups and audit arrangements before changes are applied.

In a large deduplication project, an incorrect merge rule risks affecting many customer records. Testing rules on representative data and backing up affected records helps reduce that risk. Your team should also understand which changes are reversible before approving high-impact merges.

When should you use deduplication software instead of manual checks?

Spreadsheets remain practical when your team has a small number of obvious duplicates to investigate. Larger databases introduce different challenges, particularly when records arrive regularly from multiple systems or contain inconsistent formats.

As the number of potential matches grows, repeated manual comparisons become time-consuming and harder to apply consistently. This is where data deduplication software helps. Defined matching rules are applied across the dataset, producing candidate groups that your team reviews according to its requirements.

Our Deduplication & Merge Services address these challenges through file-based cleansing, specialist data projects and CRM-specific solutions. The right approach depends on your existing systems, the complexity of the records and how frequently duplicate checks are required.

Which deduplication option is right for your database?

When your team needs to review an existing customer file, Data8 Online supports data deduplication through name-and-address matching at different levels. Its outputs are available for review, while reusable templates help keep regular data audits consistent.

For organisations carrying out regular database audits, Data8 Online supports deduplication alongside relevant cleansing and suppression checks. Reusable workflows can help teams apply consistent criteria when reviewing customer records.

Records spread across several systems often present a more complicated problem, particularly when the data needs preparing for a migration. Our Professional Services team works with organisations on these projects, developing bespoke matching and merging processes around their data.

Within Microsoft Dynamics 365 or Dataverse, duplicate records also need managing as part of ongoing CRM activity. Duplicare brings duplicate detection, prevention and merging into that environment. Its configurable rules give administrators control over the master record and the information retained during consolidation.