An AI transcription workflow should be able to produce no spoken words when a recording contains no speech. Test that explicitly before using transcripts to create meeting records or customer handovers. Pair the test with quiet, genuine speech: a filter that removes every difficult passage can stop invented text while also deleting what people actually said.
Our proposed acceptance method checks both sides of that boundary. Keep the original recording, distinguish a verified absence of speech from an unusable recording, and evaluate the transcript before it reaches a summarizer. A polished summary cannot restore evidence that was never captured.
This guide offers a fictional review exercise and a proposed test set. We did not run a speech-recognition model, generate test audio or independently test a transcription service for this article.
Check whether there are words before checking the speaker
Our meeting-transcript attribution guide separates wording, voice grouping and named identity. Here the earlier question is whether a sentence has support in the audio at all. Assigning an invented sentence to the correct meeting participant would not make it a valid record.
Consider the beginning of a recording, a muted audio channel or the interval after everyone leaves. A transcript that supplies a conversational closing in one of those intervals may look harmless. But once a later system treats every transcript sentence as observed speech, that invention can acquire a speaker, a timestamp and a place in the minutes.
Do not ask a summarizer to decide whether a sentence sounds likely. Likelihood is precisely the wrong reference. Review the audio interval and its capture status. A plausible remark remains unsupported if it was not recorded.
This is also why an empty transcript needs a status. It might mean that a valid recording contained no speech, that speech was missed, or that the recording failed. Those outcomes should lead to different actions.
What the research establishes, and what it does not
In Careless Whisper: Speech-to-Text Hallucination Harms, presented at FAccT in June 2024, Allison Koenecke and colleagues studied historical Whisper API outputs using English AphasiaBank interviews from people with and without aphasia. They found whole phrases absent from the underlying audio and an association with longer non-vocal durations.
Their detection method compared repeated transcriptions, then manually reviewed candidates. The authors acknowledge that identical inventions across runs could escape that method. They also report improvements in a later 2023 check, alongside remaining failures. This is not an estimate of the error rate of today’s meeting products, and the interview sample is not a workplace meeting sample.
The Whisper large-v3 model card also warns that predictions may contain text not actually spoken in the input. It describes repetitive output and uneven performance across languages and accents. These are documented limitations, not proof that every application using Whisper has the same behavior: segmentation, filtering and other application choices matter.
Together, these sources justify testing unsupported additions. They do not establish that a particular current provider fails, or that a particular filter solves the problem.
A fictional handover with an unsupported ending
Imagine a fictional supplier call recorded for a project handover. The last audible statement is “We still need the revised delivery date.” The recording then contains a quiet interval before it stops. In this invented exercise, a transcript adds “Everything is agreed” during that interval.
The failure is not merely an inaccurate closing. If an assistant turns that sentence into “Supplier confirmed the plan,” the downstream document now makes a commitment claim without recorded support. Even perfect speaker labeling cannot repair it.
The reviewer should mark the added sentence unsupported, preserve the actual request for a revised date and check the derived handover for the false agreement. Keep the original transcript revision available through the organization’s normal correction process; do not silently rewrite history.
Now change the exercise. Suppose the relevant channel was never captured. The reviewer cannot verify what happened during the call from that file. The appropriate outcome is “recording unavailable for this claim,” not “nobody agreed.” A later confirmation can establish a new dated statement, but it should not be represented as recovered audio evidence.
All people, words and events in this example are fictional. The point is to distinguish an invented transcript sentence from missing evidence about an actual conversation.
Build paired acceptance cases
Use approved, non-sensitive recordings with a reference established before evaluating outputs. Include cases where the right transcript contains words and cases where it does not. The following is a proposed design, not a completed test:
| Case | Reference condition | Required behavior |
|---|---|---|
| Digital silence | Valid audio file with zero-valued samples | No invented spoken words; explicit no-speech status allowed |
| Room sound without speech | Reviewed recording of background sound only | No spoken sentence attributed to a participant |
| Quiet speech with pauses | Known words spoken softly with pauses | Retain the supported words and uncertainty where needed |
| Missing capture | Expected channel absent or unusable | Report capture failure; do not infer a conversation’s content |
An environmental label such as “background noise” is different from a spoken sentence. Decide how your workflow represents such labels, so a summarizer does not quote them as participant remarks.
Add leading and trailing non-speech intervals to a speech-containing case. Check whether filtering changes the first or last word. When preprocessing cuts or combines intervals, retain a mapping back to the original recording time. Otherwise a reviewer may open the wrong interval and mistake a timing error for evidence of absence.
Record each case’s audio reference, application and model identifier where available, settings, transcript revision and expected outcome. If a hosted product hides the model or preprocessing, record that limitation instead of guessing its internal configuration.
Evaluate filters for both invention and deletion
Voice activity detection estimates which portions contain speech. It can help avoid sending non-speech regions to recognition, but it is another decision boundary to evaluate. Our recommendation is to compare unsupported additions and lost supported words separately before accepting a configuration.
In a September 2026 practitioner report, Kekoso developer Andrey Chmerev describes tests of whisper.cpp on digital silence and generated noise. He reports repeatable invented output in specific configurations and distinguishes a filter for digital emptiness from the harder case of quiet background noise. He also cautions that speech gating can clip quiet speech.
This is a developer’s account with a commercial interest, not independent validation of his app or a general ranking of model sizes. We have not replicated it. Its useful methodological prompt is to test the exact input and configuration rather than assume a named threshold proves the absence of speech.
Do not copy a loudness threshold from an unrelated recording setup and declare the workflow fixed. Microphones, gain, compression and actual voices can change what a quiet interval means. Test cases should reflect the approved capture path, including people who pause or speak softly, without inferring a medical condition from their speech.
If two runs agree, preserve that as repeatability evidence. It is not correctness evidence until compared with the recording. Likewise, fewer transcript words can indicate either fewer inventions or more omissions.
Carry the distinction into the business record
Keep separate fields for transcript text, capture status and review disposition. Suggested dispositions are supported speech, reviewed non-speech, uncertain speech and unavailable capture. They are proposed workflow categories, not a universal transcription standard.
A downstream summary should retain unresolved evidence. It may state that a decision was not established from the available recording; it should not turn an empty passage into agreement, rejection or proof that nobody spoke. Use a defined human confirmation route when the business decision cannot wait.
For a small team, start with one recurring handover and a few approved reference clips. A larger deployment can retain the same cases as regression checks when its model, capture settings or speech filter changes. Keep new examples separate from cases used to tune settings, so the evaluation can reveal failures beyond the ones already repaired.
Begin with one no-speech case and one quiet-speech case. Inspect the resulting transcript and the document that consumes it. Accept the workflow only when its handling of both absence and presence of speech matches the intended use, and record what remains unverified.