Match duplicate product or customer records across lists
A cheap matcher shortlists likely twins; Jev picks the true match or 'none', and unsure pairs go to a person.
Author's figure: CodeCut author reports 807 of 808 auto-merged pairs correct with 62 sent to review, for about $0.03 on the Abt-Buy benchmark; Southbridge reports Jev plus a small review model within 0.5 points of a frontier model at a fraction of the cost.. Reported by the builder, not measured by us.
The use case
Data and ops teams merging two product catalogs, CRM exports or donor lists get many near-duplicates that fuzzy matching gets wrong. A cheap step such as embeddings shortlists a few candidates per record. Jev is asked which candidate is the same entity, or none, with rules such as 'different model numbers mean different products'. High-probability matches merge, low ones stay separate, and the middle band goes to a person. A data steward reviews that middle band and spot-checks merges before anything is deleted, because merges are hard to undo.
Example workflow
- Export both lists (catalog CSVs, CRM or donor records)
- Shortlist the 3 closest candidates per record with Sentence Transformers or RapidFuzz
- Ask Jev to choose the matching candidate or 'none of these', with your matching rules
- Merge above a high threshold, leave below a low threshold unmatched
- Send the middle band to a person and spot-check merges before deleting duplicates
Tools
- Jev (TypeSafe AI)
- Sentence Transformers
- RapidFuzz
- Python
Risks and guardrails
- Wrong merges destroy records and are hard to reverse
- Rules must be stated clearly or matches drift
- The shortlist step can miss the true match entirely
Sources
Written by Agents Lookup. Credits and official pages for the tools this use case names:
- CodeCut article by Khuyen Tran on product record matching: codecut.ai/typesafe-jev-vs-rapidfuzz-sentence-transformers
- Southbridge.AI blog on entity resolution in campaign-finance filings: www.southbridge.ai/blog/jev-entity-resolution
- Hacker News submission (CodeCut): news.ycombinator.com/item
- GitHub repo jlink by keltokhy (record linking with Jev pair judgments): github.com/keltokhy/jlink