A note on data and privacy. Results here come from a production voice AI system serving residential property-management companies. The participating management companies are anonymized. Every caller name and street address used as an example — Kari, Whispering Pines Ct., Anika Ravi Shankar — is synthetic, chosen to illustrate a type of transcription error rather than to identify any real person.
Our voice agent answers thousands of calls a day for residential property-management companies. Homeowners call about assessments, violations, architectural requests, account access. The agent picks up, listens, and helps.
At least, that’s the design. Before it can help anyone, it has to answer a more basic question: whose account is this?
When someone calls from a phone number we have on file, that question answers itself in milliseconds. But a lot of people don’t. They’re calling from a work line, a spouse’s phone, a number they changed two years ago and never updated with their association. For those callers, the agent has to ask for a name and address and match what it hears against the account database.
We pulled 161 production calls and classified how identification went. Forty-three percent of callers weren’t pre-identified by phone number and had to go through the name-and-address lookup.
Of those, the lookup succeeded five times out of seventy.
Nineteen out of twenty callers who needed the agent to find them by name and address were not found. That single failure mode was doing more damage to the caller experience than everything else we were working on.
Why this is harder than it sounds
The problem isn’t the database lookup. The problem is that by the time the text reaches the database, it isn’t what the caller said.
Speech-to-text is very good and still wrong often enough to matter. Accents, background noise, a weak cell connection, a child in the room, someone talking while driving. A caller named Kari arrives as Carry. A street called Whispering Pines Ct. comes through as Whisperingpinescourt. Neither of those matches anything in the database, so the agent asks the caller to repeat. Then it asks them to spell it. By the time the account surfaces — if it ever does — you’ve spent two minutes of someone’s patience before answering a single question.
The obvious response is fuzzy matching. We tried the standard toolkit, and it’s worth being specific about why each piece fails, because the failures aren’t random.
Substring matching — the SQL LIKE operator — is what we started with. It looks for records containing a specific run of characters. That works when part of your input is exactly right and in the right order, which was a fine assumption when people typed their names into forms. It’s the wrong assumption when the characters themselves are wrong. LIKE '%Kari%' finds nothing when the transcript says Carry.
Phonetic matching like Soundex matches words that sound alike. It fails on the cases people actually produce. Someone who says “Bob” doesn’t match a record stored as “Robert.” A badly split address like “Breakingdotcourt” for “Breaking Dawn Ct.” either misses entirely or returns so many candidates the agent can’t choose between them.
Trigram matching — PostgreSQL’s pg_trgm — is the sophisticated option, and this is where it gets interesting. It chops each string into overlapping three-character sequences and scores the overlap. It handles spacing and word-boundary errors far better than LIKE. But it has a specific, structural weakness: each word is padded so its first letter participates in three separate trigrams. Corrupt the first letter and you destroy most of the set.
Kari against Carry shares zero trigrams. Similarity: 0.00. Evon against Yvonne shares exactly one, giving 0.09. The conventional match threshold is 0.3.
That’s not a tuning problem. Transcription errors in names hit the leading consonant constantly — it’s the least redundant part of a spoken word — so trigram similarity fails hardest on exactly the cases we most needed it to handle.
Every one of these tools assumes a kind of regularity that spoken, transcribed input doesn’t have. We needed something that could reason about how speech gets mangled.
Using the model as a query generator, not a matcher
Here’s the design decision that mattered most, and it runs against how most people use large language models for this problem.
The intuitive approach is to let the model do the matching: retrieve candidate records, hand it pairs, ask which ones refer to the same person. That cannot work inside a live phone call. An LLM call per candidate pair, at conversational latency, with someone waiting on the line — the budget is a few hundred milliseconds before people start talking over the agent. There isn’t room.
So we inverted it. Our model never sees a candidate record. It sees only the transcribed utterance, and its job is to hypothesize what the caller might actually have said.
Given the spoken name “Anika Ravi Shankar,” it produces the transcription as-is, a merged form (AnikaRavi Shankar), each component alone, and a compound-surname guess (Ravishankar). Then it compiles all of those hypotheses into a single compound database query — one round trip, not many — combining conditions with OR/AND logic. It picks between two strategies depending on what it’s looking at: a broad OR across variations for names and misspellings, or an AND across component words for addresses and partial information.
Adjudication is left to deterministic edit-distance ranking. Levenshtein distance normalized to a 0–100 similarity score, with the agent acting on the number: above 80 it treats the match as found and verifies one more detail, 60 to 79 it asks the caller to confirm, below 60 it asks for something different.
The result is one LLM call, issued in parallel with database access, doing the part of the job that requires reasoning about speech. Everything downstream is cheap and deterministic.
We also gave the agent memory across turns. When the first search doesn’t produce a confident match and it has to ask for more information, it passes along what was already tried — so the model adjusts strategy instead of repeating a failed approach.
What happened
We built a labeled benchmark from real accounts: 83 accounts across four management companies, 1,156 transcription-error variations spread across twelve distinct error types, with the correct answer known for every case.
Top-result accuracy went from 24.0% to 39.1%.
That headline hides the more useful finding, which is that names and addresses are two different problems wearing the same coat.
Addresses have structure — a number, a street, a suffix — and the approach exploits that structure well. Top-result accuracy went from 28.6% to 54.7%, a gain of 26 points that held up at every company we tested. The biggest wins came exactly where character matching had been helpless: numbers spoken as words (“one four two Maple Lane”) improved by 50 points, abbreviations by 28, spacing errors by 18.
Names are messier, and the honest result is thinner. Top-result accuracy improved only 4 points, from 19.4% to 23.4%. One error class regressed outright: homophones, where what the caller says sounds identical to the right answer but isn’t. “Georgetown Circus” for “Georgetown Circle.” This is the boundary of the approach, and it’s a clean one. Generating more spelling variations cannot help here, because the spoken and stored forms don’t differ in spelling at all — they differ in meaning while sounding the same. Query expansion solves distortions of form. It does not solve substitutions of meaning.
Latency was the obvious worry, and it didn’t materialize. Because the model call runs in parallel with database access, lookups came back at equal or better speed than before — at one company, 542 milliseconds faster.
Takeaways
The specifics here are about phone calls and property records, but the shape of the problem is common. Any time input reaches you through a lossy channel — transcribed speech, scanned documents, a search box someone typed into on a phone — you are matching a corrupted string against clean records, and the usual tools quietly assume a kind of correctness that isn’t there.
Four things to keep in mind from this experience:
Measure the failure before you design the fix. We spent an afternoon classifying 161 calls and it changed what we built. Until we had the 5-out-of-70 number, identification was one item on a list of things that could be better. Afterward it was obviously the only thing worth working on.
Treat noisy input as evidence, not as a query. The instinct is to clean up the input and then search for it. The better move is to accept that you don’t know which part is wrong, generate the plausible alternatives, and search for all of them at once.
Let the model do the part that needs judgment, and nothing else. Ours never sees a database record or decides which match wins. It generates hypotheses; deterministic code does the ranking. That split is what kept the whole thing fast enough to run inside a live conversation, and it also means the behavior you can’t easily predict is confined to one step you can inspect.
Break the metric down before you believe it. Our headline number was a 15-point gain. Underneath it were a 26-point gain on addresses, a 4-point gain on names, and one category that got worse. The aggregate would have told us we succeeded. The breakdown told us what to do next.
This work has been submitted to IAAI-27.