End-to-end latency
In one sentence
End-to-end latency is the total elapsed time across a system from one named point to another, and in voice it usually means from the moment a visitor stops speaking to the moment they hear a reply, a number that means nothing until its start and stop points are stated.
Not to be confused with Round trip time.
Definition
End-to-end latency is the total time from one point in a system to another, measured across everything in between.
In voice it usually means from the visitor finishing speaking to the visitor hearing a reply, though the phrase is used loosely, which is where most of the trouble comes from.
The term is only meaningful once the two endpoints are stated. The most common source of confusion in latency discussions is two parties measuring different spans while using the same words.
The candidate definitions, all of which appear in practice
- Speech end to audio start. The most useful for conversational feel, and the same span as time to first audio.
- Speech end to response complete. Includes the whole spoken reply, which is far longer and rarely the right measure.
- Request received to response begun. A server-side measurement that leaves out the network and the client entirely.
- Round trip including client processing. The most complete span, and the hardest to instrument.
Why the distinction matters commercially
- A vendor measuring server-side and a customer measuring speech end to audio start will disagree by hundreds of milliseconds, and both will be reporting honestly.
- A service level agreement that specifies end-to-end latency without defining the endpoints is unenforceable in practice.
The full span in a voice pipeline
- Endpointing, typically 150 to 300 milliseconds.
- Upstream transport.
- Transcription finalization.
- Retrieval, where the system uses it.
- Model time to first token.
- Synthesis time to first byte, typically 100 to 200 milliseconds.
- Downstream transport and the start of playback.
Where teams look first, and should not
- The model is the most visible component and frequently not the largest contributor.
- Endpointing alone can consume close to a third of a one-second budget.
- Retrieval degrades silently as the corpus grows.
- Network contribution is invisible in local testing and substantial in production.
Instrumentation is the actual requirement
- Latency cannot be improved without per-stage measurement.
- A single end-to-end number tells you there is a problem and nothing about where it is.
Common misconception
That end-to-end latency is a single well-defined metric. It is a span whose endpoints have to be specified, and unqualified use of the term is the source of most latency disputes.
Why it matters commercially
This is the number a buyer feels and the number a contract should specify. Getting the definition into the contract, with the measurement point named, is worth more than negotiating the target value, because a target with no defined start and stop point cannot be enforced.
In voice specifically
In text the reply can stream onto the screen and the reader tracks progress as it fills, so a loose latency figure rarely causes an argument. In voice there is nothing to watch, so the exact span being measured is the whole of what a listener feels, and two honest numbers for the same call can differ by more than a second depending on where the clock was read.
Where AsqVox fits
For an AsqVox widget the span that matters is the one the visitor lives through: from the moment they stop speaking to the moment they hear a reply, measured in the browser. A client-side number is the only one that reflects the network and the device the visitor actually has, which is why it is the span the widget is built to answer for.
Visual
Four spans, one phrase
Specify the endpoints, or the number means nothing.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
Production budgets commonly allow 150 to 300 milliseconds for endpointing, the first stage of the span, before any processing begins.
150 to 300 msindustry rangeCommon production budgeting convention, 2026 - An engineering convention, not a research finding. At the top of the range, endpointing alone is close to a third of a one-second budget, which is why the model is rarely where to look first.
Streaming synthesis time to first byte typically lands at 100 to 200 milliseconds, near the end of the span.
100 to 200 msindustry rangeCommon production budgeting convention, 2026 - Also an engineering convention, and it applies to streaming synthesis only.
Model time to first token, independently measured across leading realtime models, runs from around 0.78 seconds for xAI Grok Voice to around 2.98 seconds for Gemini 3.1 Flash Live.
0.78s to 2.98sindependentArtificial Analysis, via Softcery, April 2026, 2026 - This is one span inside the pipeline, not the whole of it. The near four-fold spread means model choice on its own can decide whether the end-to-end number clears a second.
Sub-second time to first audio is the working design target for conversational voice.
Under 1,000 msindustry rangeIndustry working threshold, 2026 - A working convention, and one that only means something once the measurement span is named.
Past roughly 1,500 milliseconds, a conversation is widely reported to feel broken.
1,500 msindustry rangeIndustry working threshold, 2026 - A shared working line rather than a measured cliff.
Human conversational turn gaps average around 200 milliseconds, the baseline every design target is set against.
200 msindustry rangeWidely cited conversational turn-taking baseline, 2026 - A reference point rather than a target, quoted widely enough that no single study owns it.
There is no industry-standard definition of end-to-end latency for voice agents, and no standard measurement point.
-no reliable figureA genuine gap that makes vendor comparison difficult, and one worth stating out loud, because it is what lets two honest numbers for the same call differ by more than a second.
There is no published benchmark of end-to-end latency across deployed voice agent platforms under matched conditions.
-no reliable figureThe independent public measurements cover time to first token, one component of the span. The only reliable end-to-end figure is one you measure yourself, on your own stack.
Examples
In practice
A vendor and a customer dispute service level compliance. The vendor measures from request received to first synthesized byte and reports 620 milliseconds. The customer measures from speech end to audible playback and reports 1,780 milliseconds. Both are accurate. The contract specified end-to-end latency without defining the endpoints, and the dispute is unresolvable on the wording.
The everyday version
End-to-end latency is the whole wait, from your customer finishing their sentence to hearing a reply. The catch is that everyone measures a slightly different stretch of it. When someone quotes you a latency number, ask where they start and stop the clock.
Usage
Who says it
- Engineers, buyers and vendors, frequently meaning different things by it.
- It appears in service level agreements, where the definition is essential and often absent.
Where it turns up
- Next to latency targets, percentile requirements, measurement methodology and the remedy for a breach.
- A well-drafted RFP defines the measurement point. A poorly drafted one produces bids that cannot be compared, because every vendor measures from the point that flatters it most.
Common misuse
- Using the term without specifying the endpoints.
- Reporting a median without percentiles.
- Diagnosing latency without per-stage instrumentation.
Questions people ask
What is end-to-end latency in voice AI?
It is the total time from one point in a system to another, measured across everything in between. In voice it usually means from the visitor finishing speaking to the visitor hearing a reply, which is the same span as time to first audio. The term is only meaningful once the two endpoints are stated, because several different spans travel under the same phrase.
Why do a vendor and a customer report different latency numbers for the same call?
Because they are measuring different spans. A vendor often measures server-side, from request received to first synthesized byte, while a customer measures from speech end to audible playback. In one dispute the vendor reported 620 milliseconds and the customer 1,780 milliseconds, and both were accurate. When the contract does not define the endpoints, the disagreement cannot be resolved on the wording.
How should end-to-end latency be specified in a contract?
Name the measurement point: where the clock starts and where it stops. For a voice agent, speech end to audio start is usually the right span, because it is what the visitor feels. Quote percentiles rather than a median, since the slow calls decide how a product is judged. A target with no defined endpoints is unenforceable, so the definition is worth more than the number.
Why is the model usually not the cause of high latency?
Because it is the most visible component and frequently not the largest. Endpointing alone can consume close to a third of a one-second budget, retrieval degrades silently as the corpus grows, and network time is invisible in local testing but substantial in production. A single end-to-end number tells you there is a problem and nothing about where, so per-stage instrumentation is the actual requirement.
Last reviewed 31 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.