Percentile latency describes how latency is distributed rather than averaged, so a p95 figure means 95 percent of requests were faster than it and the slowest 5 percent, the turns a visitor actually remembers, were slower.
0.78s to 2.98s Artificial Analysis, via Softcery, April 2026Last reviewed 31 July 2026
Definition
Percentile latency describes how latency is distributed instead of averaging it away. A p95 of 2,400 milliseconds means 95 percent of requests came back faster than that, and the slowest 5 percent did not.
It is the standard way to report latency, because the average hides exactly the behavior that decides whether a voice product is acceptable.
An average tells you neither the typical experience nor the bad one. A handful of very slow requests drag the mean upward without describing anyone in particular, which is why percentiles, not averages, are how latency gets reported.
The common percentiles, and what each one tells you
p50, the median. Half of requests are faster than this. It is the typical experience.
p90 and p95. The edge of normal, where noticeable slowness begins.
p99, the tail. One request in a hundred is this slow or slower.
Why the tail matters more in voice than on the web
A slow web page is mildly annoying. A slow voice response is a silence, and during it a person decides the system has failed and starts talking again.
Across a ten-turn conversation, a p95 event is something the visitor is likely to run into at least once. The tail is not rare from where they sit; it is the thing they meet.
A system with an excellent p50 and a poor p99 feels like an agent that works well and occasionally freezes, which erodes trust faster than something that is uniformly mediocre.
Why chaining stages makes the tail worse
A voice pipeline runs several services in sequence, and the latency of a turn depends on all of them.
If each stage has an independent chance of being slow, the odds that at least one of them is slow climb with the number of stages.
So the end-to-end p99 comes out worse than any single component's p99, sometimes by a lot. Component-level percentiles are necessary but not sufficient.
What to put in an agreement
A percentile, a target, a measurement point and a measurement window.
p95 is a common contractual choice. A p50 on its own is not a meaningful commitment.
Without a measurement window, a target can be met on the monthly aggregate while failing badly during peak hours.
Common misconception
That the average latency is a useful summary of it. It is close to the least useful single number available, because it describes neither the typical experience nor the bad one.
Why it matters commercially
Percentiles are how latency gets written into a contract and how quality actually gets experienced. A vendor who reports only averages is either unsophisticated or quietly hiding a tail, and either way the number to ask for is p95 with a measurement window attached.
In voice specifically
On the web the tail is a slow page and a spinner. In voice the tail is a silence with nothing on screen, and a person fills it by concluding the agent has frozen and talking over it. That is why the worst 5 percent of turns, not the median, is what a voice product gets judged on.
Visual
The average describes nobody
Voice quality is decided by the worst turns, not the average one.
The four points are the illustrative distribution from the worked example below, not a measured benchmark. The bars underneath are the chance a visitor meets at least one p95 turn as a conversation runs longer, worked out as one minus 0.95 to the power of the turn count: roughly 40 percent across ten turns, so the tail is not a rare edge case from the visitor's side. Chain several pipeline stages and the end-to-end p99 comes out worse than any single stage's p99. Specify a percentile, a target, a measurement point and a window, because a monthly aggregate can pass while peak hours fail.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
Sub-second time to first audio is the working design target for conversational voice, which is the line the p50 sits comfortably under and the tail routinely crosses.
Under 1,000 msindustry range
Industry working threshold, 2026 - A shared working convention rather than a published standard. It is the number the median is supposed to clear.
Past roughly 1,500 milliseconds a conversation is widely reported to feel broken, which is why a p90 or p95 sitting above that line matters even when the median looks healthy.
1,500 msindustry range
Industry working threshold, 2026 - A working line, not a measured cliff. A visitor feels it without being told to look for it.
Human conversational turn gaps average around 200 milliseconds, the baseline every latency target is set against.
About 200 msindustry range
Conversational turn-taking baseline, 2026 - A reference point, not a target you built. It is the pace the median is trying to feel like.
Independent measurements put model time to first token across leading realtime models between roughly 0.78 and 2.98 seconds, and these are typically published as central figures rather than as distributions.
0.78s to 2.98sindependent
Artificial Analysis, via Softcery, April 2026, 2026 - Cite it as a central-figure comparison. Because the numbers are not distributions, they say nothing about the tail, which is the part percentile latency exists to expose.
Voice AI providers rarely publish percentile latency distributions, reporting central figures instead, which makes independent tail assessment impossible from published material.
-no reliable figure
A genuine transparency gap in the category. The absence of a p95 in a vendor sheet is itself information.
There is no standard measurement window convention for voice latency service level agreements, so two providers can both claim to meet a p95 target while measuring it over very different windows.
-no reliable figure
Which is why a percentile target without a stated window is close to unenforceable.
Examples
In practice
A deployment reports a median time to first audio of 850 milliseconds and starts getting complaints that the agent freezes. Percentile analysis shows p95 at 2,400 milliseconds and p99 at 4,100, driven by cold starts on a retrieval service that is invoked infrequently. The median was fine the whole time. Warming the service flattens the tail and the complaints stop, with no change to the median at all.
The everyday version
Percentile latency means looking at the slowest responses rather than the typical one. If nineteen replies out of twenty are quick and one takes four seconds, the average still looks fine and your customer remembers the four-second one. In a short conversation they will almost certainly land on it.
Usage
Who says it
Infrastructure and reliability engineers, as a matter of standard practice.
p95 latency is the value that 95 percent of requests came back faster than, with the slowest 5 percent slower than it. It describes the edge of normal experience rather than the typical one, which is why it is a common choice for a service level commitment. A p50, the median, tells you about the typical turn; a p95 tells you about the ones a visitor actually remembers.
Why is average latency a bad measure?
Because it describes neither the typical experience nor the bad one. A small number of very slow requests drag the mean upward without matching anyone in particular, so an average can look healthy while a real tail is making the product feel broken. A vendor who reports only averages is either unsophisticated or hiding that tail.
Why does the latency tail matter more in voice?
Because a slow voice response is a silence, and during it a person concludes the system has failed and starts talking again. Across a ten-turn conversation there is roughly a 40 percent chance of hitting at least one p95 turn, so the tail is not a rare edge case from the visitor's side. A system with a great median and a poor p99 feels like an agent that works well and then freezes, which costs more trust than being uniformly mediocre.
What should a voice latency SLA specify?
Four things: a percentile, a target, a measurement point and a measurement window. p95 is a common contractual choice, and a median alone is not a meaningful commitment. Without a window, a target can pass on the monthly aggregate while failing badly during peak hours, so the window is as important as the number.
Share this definition
Last reviewed 31 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.