15 August 2026
Meeting transcription is two separate machine-learning steps run one after the other: a speech recognizer turns sound into words, then a diarizer decides which voice said them. Everything you see afterward — notes, decisions, search — is built on that text and inherits whatever either step got wrong. Neither step tells you when it failed; the output just reads as a normal sentence.
There are four stages, and each one only sees the output of the one before it. Audio capture turns sound waves into a digital stream. A recognizer (the ASR model) turns that stream into text. A diarizer attributes chunks of that text to a speaker. Notes, decisions, and search are then generated from the attributed text.
No stage gets to see the original audio again once it hands off. If the recognizer misheard a word, the diarizer attributes the wrong word to a speaker with total confidence — it has no way to know the word was wrong, because it never had the audio to check against. Errors don't cancel out downstream. They compound.
Before any AI model runs, the microphone and network have already decided how much information survives. This is the stage most people never think about, and it's often responsible for more word errors than the recognizer itself.
Recognition can run in two places, and they trade off differently. A browser-side model runs on the device in the tab, so audio never has to leave the machine and it keeps working on a bad connection — but it's usually a smaller model with a narrower vocabulary, so it does worse on jargon, product names, and accents it wasn't trained heavily on.
A server-side model sends audio to a larger model running elsewhere, gets better vocabulary coverage and accent handling in return, and pays for it with round-trip latency and the fact that audio now has to leave the device during the call. Two tools transcribing the identical audio from the identical meeting can output genuinely different text for this reason alone — not because one is broken, but because they're different models trained on different data making different guesses at the same ambiguous sound. AVAY runs recognition server-side during a live call, which is why it can hold a broader vocabulary and answer questions on the spot — the trade-off is that it needs a live connection to its transcription service rather than working fully offline.
Diarization is a separate model from the one that recognizes words. It groups short windows of audio by voice characteristics — pitch, cadence, spectral shape — into clusters, then maps each cluster to a name using whatever identity signal is available, usually the account someone joined with.
This fails on its own terms, independently of whether the words were transcribed correctly. Two people with similar voice registers can get merged into one cluster. Someone talking over another speaker produces a window with two voices in it, and the model has to guess which one to credit. A person who switches from a laptop mic to AirPods mid-call can get split into two different 'speakers' for the rest of the transcript. The words can be perfect and the attribution still wrong.
A human note-taker who mishears something usually asks you to repeat it. A recognizer never does — it outputs whichever word sequence it judged most probable and renders it as a complete, grammatical sentence. 'Clarify the number' and 'modify the number' are both plausible five-word sentences; the model picks one and the transcript shows no hesitation either way.
Most interfaces don't surface confidence scores even when the underlying model produces them, so a correctly transcribed line and a wrong one look identical on screen. Nobody notices until a specific number, name, or commitment turns out to be wrong days later — by which point it's already been copied into a note or a decision log.
Notes summarize the transcript, so a misheard word becomes a wrong fact in the summary with no flag attached. A decision log records who committed to what, so a diarization error at minute twelve becomes the wrong name attached to a real commitment. Search matches on the text that was actually recorded, so if a term was misheard once, that meeting simply won't surface when you search for the correct term later.
AVAY builds its live notes and decision log from this same transcript stream as the meeting happens, which is why it can keep them current without a separate pass afterward — but it also means an attribution error isn't caught by a review step before it lands in the decision log. Treat anything downstream of transcription as inheriting its accuracy, not improving on it.
| What you gain | What you lose | |
|---|---|---|
| Browser-side (on-device model) | Audio never leaves the device; keeps working on a weak connection | Smaller model — weaker on accents, jargon, and proper nouns |
| Server-side (cloud model) | Larger model with broader vocabulary and better accent handling | Audio has to leave the device; needs a live connection |
Diarization and word recognition are separate models. If two people sound similar or talk over each other, the diarizer can attribute perfectly correct words to the wrong speaker — that's a clustering error, not a hearing error.
Usually you can't tell from looking at it. The recognizer outputs whichever word sequence it judged most probable and renders it as a normal, grammatical sentence whether it's right or not. Few interfaces show the confidence score behind each word, so a misheard word looks identical to a correct one.
They're likely running different models on different amounts of audio context — a browser-side model versus a server-side one, or two vendors' recognizers trained on different data. Both can be reasonable guesses at ambiguous audio and still disagree, especially on names and acronyms.
Usually yes. A strong recognizer defeated by cross-talk, a bad mic, and a weak connection will do worse than a mediocre recognizer given clean, single-speaker audio. Capture quality sets a ceiling the model can't recognize its way past.
Transcription is two stacked, independently fallible steps — turning sound into words, then words into a speaker — and every note, decision, and search result downstream inherits both errors without any flag showing which parts to trust.
Meetings that take their own notes, in the browser: avay.ai.