# Matching rules

How the profiler finds duplicate candidates, how bands are assigned, and how to calibrate before anyone merges. A band is evidence for a reviewer, never permission to merge.

## Never sample for duplicates

- Duplicates need the whole object. In a 1,000-row sample of 100,000 records, both records of a given pair are present about (1,000 / 100,000) squared of the time: 0.01%. A sample finds almost no pairs and reports a clean CRM.
- If X08 says the export is PARTIAL, duplicate counts are a floor, labelled as such.

## Normalisation per field

- Text: Unicode NFKC for display; for match keys NFKD with accents removed, lowercase, "&" to "and", punctuation removed, spaces collapsed.
- Person names: honorifics (mr, mrs, ms, miss, dr, prof) and suffixes (jr, sr, ii, iii, iv) removed. Full name is first plus last. If either name holds letters outside the Latin script, only exact equality counts: Salesforce documents that its fuzzy methods support only Latin characters and recommends exact matching for international data.
- Email: trimmed, zero-width characters removed, whole address lowercased. RFC 5321 lets hosts treat the local part as case-sensitive but discourages relying on it, so case-only differences count as the same mailbox; say so in the report. Variant key: plus tag removed; dots removed in the local part for dot-insensitive domains (default gmail.com and googlemail.com).
- Phone: digits only. Invalid below 7 or above 15 digits (E.164 allows at most 15). Placeholder when every digit is the same or the digits are an ascending or descending run. Key: last 10 digits.
- Domain: from the domain, website or email column; scheme, credentials, port, path and query stripped; lowercase; leading "www." removed. Registrable domain is the last two labels, or three when the last two are a common two-level suffix (co.uk, org.uk, ac.uk, gov.uk, com.au, net.au, org.au, co.nz, co.jp, com.br, com.mx, co.in, co.za, com.sg). This is not the full Public Suffix List; check odd suffixes by hand.
- Company names: text normalisation, then legal-form tokens removed from the end only, repeatedly (inc, corp, co, company, llc, llp, lp, ltd, limited, plc, gmbh, ag, kg, sa, sas, sarl, srl, spa, bv, nv, oy, ab, as, aps, pty, pte, kk), and a leading "the". Salesforce's standard company normalisation also removes suffixes such as Inc, Corp, Limited and Ltd.

## Similarity and thresholds

- Jaro-Winkler, prefix scale 0.1, prefix up to 4 characters, scored 0 to 1. For persons the score is the higher of the full name and the name with tokens sorted, which catches first and last swapped (Salesforce's standard rule also transposes them).
- Defaults: person probable 0.92 with a corroborating key, person possible 0.88 with the same company name, company probable 0.93.
- Why higher than Salesforce's published standard-rule thresholds (first name 85, last name 90, company 70, city 85 on a 0 to 100 scale): the script uses one similarity method on the whole name, without Salesforce's per-field weighting, so it starts stricter and lets calibration loosen it. Those thresholds are on Salesforce's standard matching rule pages (checked 2026-11-06). An org can run its own rules, so read them in Setup.
- Reference values: "martha" and "marhta" score about 0.961; "jordan smith" and "jordan smyth" about 0.967; "sam lee" and "samuel lee" about 0.94.

## Blocking keys

Only records that share a block are compared. Measured on 2026-11-01 with synthetic data: about 50 seconds for 103,000 contacts on a laptop ([profiler.md](profiler.md)).

- Persons: unique key; each normalised email (primary and additional); email variant key; phone key; first initial plus the phonetic code (Soundex) of the last name, so Smith and Smyth meet; email registrable domain plus that phonetic code; name tokens sorted.
- Companies: unique key; registrable domain (personal domains excluded); first 6 characters of the normalised name plus country; phone key.
- Deals: unique key; associated company plus pipeline.

## Bands and rules

- Certain: same unique key (D01) or same mailbox (D02).
- Probable: plus-tag or dot variant (D03); similar name with a corroborating key (D04); same domain with a similar name, same name and phone, or same name and country when the two records do not hold different domains (D05); two open deals at one company in one pipeline with amounts within 5% created within 30 days (D06).
- Possible: same company name with a similar person name (D04); shared domain with different names, same name in the same country with different domains, or a similar name in the same country with a domain missing (D05); same open deal name at one company (D06).
- Two companies with the same name and different domains are two companies until shown otherwise: generic names such as Summit Consulting collide inside one country.
- A pair takes its strongest band; a key conflict then caps it. Each pair carries a basis (for example "D05 same name and country"), which names the evidence behind its band in calibration-sample.csv and possible-pairs.csv.
- The probable band of D03, D04, D05 same name and country, D05 same name and phone and D06 can each be set to possible in the rules file (`d03_band`, `d04_band`, `d05_name_country_band`, `d05_name_phone_band`, `d06_band`) when calibration says so.

## Guards

- Blank and placeholder values never form blocks or groups. A dedupe that groups on email alone puts every record without an email in one group: HubSpot's official crm-data-quality skill groups contacts on identical email and merges into the lowest ID, so feed it a reviewed plan instead.
- Role mailboxes never link people; a shared role mailbox is INFO.
- Converted Salesforce leads never enter the duplicate engine. A person who comes back as a new lead shows up as D07 against the contact the old lead converted into, and the fix is to convert the new lead into that contact.
- Personal mail domains never corroborate a company match.
- A phone shared by 4 or more different names (a switchboard) and any block above 50 records are reported once as D08 and never compared. When the oversized block is a unique key or an email, D08 and D01 or D02 warn: the value is not identity evidence, it is a broken key.
- Different non-blank unique keys cap the pair at possible and set key_conflict. Two records claiming different billing accounts are different source entities until the account boundary says otherwise.
- Groups come from certain and probable links (union-find), certain links first, then by score. A group is certain only if every link inside it is certain. Possible links stay as pairs, never grouped.
- A union that would put two different unique keys in one group is refused; the bridging link becomes a possible pair marked refused. Without this, a record with no key that matches records holding PU-1 and PU-2 would chain two people into one merge group.
- Any group above 10 records is chain risk (D09): split it by hand. Twelve contacts sharing a "unique" key of TEMP means the key is not unique; demote it in the rules file and rerun.

## Account boundary and shared domains

- Decide the account boundary first (legal entity, billing account or business unit) and record it in `decisions.md`. Without it, every shared-domain pair stays "investigate" and gets no merge plan.
- Subsidiaries, business units and reseller customers legitimately share domains. A shared domain is a reason to look, not a match.
- When two companies share a domain and hold different billing keys, keep them separate and fix contacts' associations instead.

## Calibration

- calibration-sample.csv (default `--calibration 20`) holds up to 20 pairs per band per object, in a stable order seeded by `--as-of`, with each pair's rule IDs and basis. A rerun into the same folder keeps the marks already typed for every pair it still samples.
- A person with CRM access marks each pair same, different or unsure in reviewer_mark, from the CRM record pages, not the export. The agent may pre-fill marks from read-only MCP reads; the person confirms them before any band feeds the merge plan. Request only the record ID and the fields under check where the tool lets you choose properties, compare name, email and phone values inside the tool result, and write only the mark.
- Pass: certain 20 of 20 (or all, if fewer); probable at least 18 of 20 (90%). Unsure does not count as same, so look again at unsure pairs before tightening anything.
- Certain below the pass mark: a D01 key is not unique, so demote it in the rules file; a D02 mailbox is shared by different people, so mark the pair different with the note "shared mailbox", keep it out of the merge plan and ask the record owner to split the address. Pairs noted this way leave the certain count.
- Probable below the pass mark: count the different marks by basis and tighten only each basis where more than 1 in 10 of its sampled pairs is different:
  - D03 plus tag or dots removed: set `d03_band` to possible.
  - D04 similar name with a corroborating key: raise `name_similarity_probable` by 0.02 when the false matches score below 1.0. When they are identical names (score 1.0, common for two people with one name at a large account), no threshold separates them: set `d04_band` to possible.
  - D05 same domain, similar name: raise `company_similarity_probable` by 0.02.
  - D05 same name and country: set `d05_name_country_band` to possible. D05 same name and phone: set `d05_name_phone_band` to possible (branches that share a central number).
  - D06: halve `deal_amount_tolerance` and `deal_created_window_days`; if it fails again, set `d06_band` to possible.
- Rerun with the same `--as-of` and mark only the pairs without a mark. Pairs from the bases you did not touch keep their marks. Record the marks, the precision per band and each threshold change, with its `decisions.md` entry, in the report.

## Cross-check with the CRM's own duplicate tools

- HubSpot duplicates manager compares contacts on first and last name, email, IP country, phone, postal code and company name, and companies on domain, name, country, phone and industry; it recalculates as records are created and daily. Individual review needs Professional or Enterprise (up to 10,000 pairs); bulk and custom rules need Data Hub (Professional 30,000 pairs, Enterprise 100,000); custom rules allow two per object and nine properties.
- Salesforce standard contact and lead matching rules combine name with email, phone, title and company, or street; match keys narrow candidates to the 100 most likely before scoring; records saved in the same operation are not compared with each other by block or alert rules; duplicate jobs (Performance and Unlimited) scan the whole org.
- Attio prompts on matching company domains and person emails; people's email addresses and companies' domains are unique attributes; deals have no unique attribute by default.
- HubSpot's rejected duplicate pairs can be restored for 14 days. A rejection is a recorded identity decision, not a deletion; keep the reason.
- Read the comparison: pairs found by both are strongest; pairs only the CRM found point to a field missing from the export or a normalisation gap; pairs only the script found point to the CRM rule's scope or edition limits.