Voice activity detection
In one sentence
Voice activity detection (VAD) is the component at the very front of a voice agent that decides, frame by frame, whether the incoming audio contains speech or only background noise.
Not to be confused with Endpointing, or Turn detection.
Definition
Voice activity detection is the gate at the front of a voice agent. It listens to the incoming audio and decides, continuously, whether anybody is talking.
It does not know what was said, and it does not know who said it. It knows only that there is speech in the room.
The component works on short audio frames, usually 10 to 30 milliseconds each, and returns a speech-or-not verdict on every one of them. Older implementations read energy levels and spectral features. Those were cheap to run and easy to fool, since any loud enough noise looked like a voice. Neural detectors replaced them: still small, still fast, and far harder to trick, able to hold speech apart from music, typing, traffic and other people at very little computational cost.
What the gate is actually for
- Deciding when to start streaming audio to the recognizer at all, which keeps cost and bandwidth off long stretches of silence.
- Feeding the endpointing logic, which is what turns a speech-or-not signal into a decision that the turn has ended.
- Spotting an interruption while the agent is talking, which is the first link in the barge-in chain.
- Keeping background noise out of the transcript. Without that, the system opens phantom user turns full of nonsense and the conversation stops making sense.
Two failure modes, and their costs point opposite ways
- False negative. Speech gets read as silence, so the agent misses the visitor entirely or clips the first words off their sentence. It comes across as deafness.
- False positive. Noise gets read as speech, so the agent believes it is being interrupted, stops mid-answer, or writes down something nobody said. It comes across as twitchiness.
- Tuning lives on that line, and the right setting depends on where the audio is coming from. A contact center agent and a website widget open on a phone in a cafe do not want the same threshold.
Three things it is not
- Not recognition. It registers that speech is present, never what the words were.
- Not turn detection. Hearing that speech has stopped is a different question from knowing the person has finished, and that question gets its own page.
- Not speaker identification. It cannot tell you who is talking, or even whether the voice belongs to the person the agent is meant to be serving.
Common misconception
That detection is where the latency goes. It is not. A neural detector is extraordinarily cheap to run, and when a voice agent feels slow this is almost never the reason. Time spent tuning it for speed is usually time spent on the wrong stage of the pipeline.
Why it matters commercially
Tuning here is invisible when it is right and maddening when it is wrong. Most complaints that an agent keeps cutting people off, or cannot hear them, trace back to detection and endpointing settings rather than to the quality of the model. It is a small component with an outsized effect on how intelligent the system seems.
In voice specifically
A text box has a send button, so the system is told when the message is finished. Speech has no send button. That is why something has to sit at the front and judge, frame by frame, whether anybody is talking at all.
Where AsqVox fits
A website widget works in a harder listening environment than a phone line. Browser audio arrives from unknown devices in unknown rooms, and the agent's own voice is frequently audible through the visitor's laptop speakers.
Visual
The gate at the front
VAD knows there is speech. It does not know the sentence is finished.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
Silero VAD scores a 30 millisecond audio chunk in under 1 millisecond, which confirms that voice activity detection is not normally a meaningful contributor to end-to-end latency.
Under 1 ms per 30 ms chunkvendor claimSilero VAD published benchmark, 2026 - Published by the project itself. The useful conclusion is directional: when a voice agent feels slow, detection is rarely where to look first.
Endpointing, which consumes the detection signal, typically waits 150 to 300 milliseconds of trailing silence before declaring a turn over.
150 to 300 msindustry rangeCommon production budgeting convention, 2026 - An engineering convention drawn from production practice rather than a published research finding. It is the adjacent setting most often mistaken for a VAD setting.
Barge-in should take effect within roughly 60 milliseconds of a detected interruption.
60 msindustry rangeIndustry working threshold, 2026 - Detection is the first link in that chain and is not the part that consumes the budget. Verification, flushing and playback control are.
The average gap between turns in human conversation is around 200 milliseconds, which is the reference every threshold downstream of detection is set against.
200 msindustry rangeWidely cited conversational turn-taking baseline, 2026 - A reference point rather than a target. It is quoted widely enough that no single study owns it.
No standard published benchmark exists for voice activity detection accuracy in realistic noisy consumer conditions, and there is no common test set that vendors report against. Comparative claims about how well one detector copes with noise are generally unverifiable.
-no reliable figureState this plainly when a supplier claims better noise handling than a rival. There is no third-party scoreboard to check the claim against, so the only evidence available is a trial on your own traffic.
Examples
In practice
A widget on a mobile-heavy site keeps producing truncated visitor turns. The logs show detection firing on the visitor breathing and on music playing behind them, opening turns that contain no words at all. Raising the speech probability threshold, and requiring a minimum run of speech before a turn can open, removes the phantom turns with no measurable increase in clipping.
The everyday version
This is the part deciding whether a person is speaking to the agent or a door has just banged shut. Err one way and the agent will talk over your customer. Err the other way and it will sit in silence while they speak.
Usage
Who says it
- Voice and audio engineers use it constantly. It is a standard component name and needs no translation among them.
- It almost never comes up in a sales or buying conversation, and it does not need to.
- It surfaces occasionally in support escalations, once a complaint has been traced back to its actual cause.
Where it turns up
- In architecture documents and integration guides rather than in RFPs.
- In configuration interfaces, where it shows up as tunable parameters: minimum silence duration, minimum speech duration, and the speech threshold itself.
Common misuse
- Using voice activity detection interchangeably with endpointing or turn detection. It is an input to both, not another word for either.
- Blaming model quality for behavior that is a detection tuning problem. This is the most common misdiagnosis in an early voice deployment.
Questions people ask
What is voice activity detection used for?
Four jobs. It decides when to start streaming audio to the recognizer, which keeps cost and bandwidth off silence. It feeds the endpointing logic that declares a turn finished. It spots the visitor starting to talk over the agent, which is what makes barge-in possible. And it keeps background noise out of the transcript, which otherwise opens phantom user turns that make the conversation incoherent.
What is the difference between VAD and endpointing?
Voice activity detection returns a speech-or-not verdict on every short frame of audio. Endpointing is the rule applied on top of that signal: if speech has been absent for a set number of milliseconds, the turn is over. One is a detector, the other is a decision, and treating them as synonyms is the most common mistake with the term.
Does voice activity detection add latency?
Almost none. Silero VAD scores a 30 millisecond audio chunk in under 1 millisecond, so when a voice agent feels slow this is rarely the reason. Look at endpointing, retrieval and the model instead. Teams that spend their effort optimizing detection are usually optimizing the wrong stage.
Why does my voice agent keep cutting me off, or fail to hear me?
Both symptoms usually come from the same place, and it is not the language model. Too sensitive and the agent reads noise as an interruption, so it stops mid-answer and feels twitchy. Not sensitive enough and it reads speech as silence, so it misses you or clips your opening words and feels deaf. The correct setting depends on where your visitors actually are, which is why a call center and a browser widget need different ones.
Last reviewed 30 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.