Concurrency
In one sentence
Concurrency counts the conversations a voice system can run at once, which makes it a question about capacity rather than about volume, and it governs what happens on the busiest day instead of the ordinary one.
Definition
Concurrency counts how many conversations a voice system can run at once. It is the capacity question, and it settles what happens on your busiest day.
Spread a thousand conversations across a working day and nothing strains. Land a hundred of them inside two minutes and you have an incident.
The unit is simultaneous active sessions, and that measures something else entirely from how much traffic arrives over a period. Treating one figure as an answer to the other is how a system that looked comfortably provisioned goes down on the one day anyone was watching.
Where concurrency limits actually come from
- Quotas on the recognition, language and synthesis APIs, written as requests per minute, tokens per minute or caps on concurrent streams. More often than not this is the real ceiling.
- Channels on a SIP trunk, if there is one, which is capacity you bought in a fixed quantity.
- Whatever handles the audio itself, including echo cancellation.
- Everything downstream. A booking API good for ten requests a second caps the whole system, however much voice capacity is stacked in front of it.
How demand actually arrives
- This traffic comes in bursts, and the bursts track things happening elsewhere. Send a campaign, post an outage notice, launch something or get mentioned in the press, and the curve spikes rather than climbs.
- What matters is the ratio between peak and average, not the average. Size the system for the average and you have guaranteed a failure at the peak.
- Erlang models came out of old telephony traffic engineering and are still where a telephony capacity plan should start.
Degradation behavior, which is the part usually left unspecified
- Reject. Turn new sessions away cleanly and say why. Honest, and nobody enjoys it.
- Queue. Make callers wait, which only works with a queue experience and a stated wait time.
- Degrade. Drop to a cheaper and faster model, or to text, and keep serving at reduced quality.
- Fail. Sessions snap mid-conversation. The worst of the four, and what you get by default when nobody picked.
- Picking among them belongs in the product decisions. It rarely gets made there.
Cost coupling
- Spend here tracks usage, so any surge in traffic is also a surge in the bill. Rate limiting is a budget control wearing an engineering hat.
- Stack everything together and production voice runs somewhere around USD 0.11 to USD 0.33 a minute, which is what makes an uncapped spike hurt.
Common misconception
That only large companies need to think about it. One post that travels can push a small business past its ceiling inside a few minutes, and it breaks at precisely the moment the most people are looking.
Why it matters commercially
Procurement asks about it as standard and often writes the answer into the contract. It is also how a serious buyer tells a platform from a demo, because giving a real answer means knowing exactly where your own architecture stops.
In voice specifically
A queued text chat annoys someone. A voice session that dies mid-sentence usually ends the conversation for good, since there is no thread on screen to scroll back to and no obvious way to resume. Degradation planning carries more weight in voice than it ever does in chat.
Where AsqVox fits
There is no trunk to buy for a widget on a website, so what binds is model capacity upstream and the audio processing. The Orb's text chat fallback is a genuine degradation route: when load climbs, dropping to text still answers the visitor rather than dropping them.
Visual
Volume is not capacity
Peak decides your capacity. Average decides your bill.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
Base platform rates as published sit near USD 0.05 a minute for Vapi, USD 0.07 for Retell and USD 0.09 for Bland.
USD 0.05 to 0.09 per minvendor claimVendor published pricing, 2026 - Platform rate only, and rate cards move. It earns a place here because usage-based pricing turns any capacity surge into a billing surge at the same time.
Add recognition, the model, synthesis and telephony together and real production cost settles somewhere near USD 0.11 to USD 0.33 a minute.
USD 0.11 to 0.33 per minindustry rangeWidely cited industry range, 2026 - Nobody audited this; it points in a direction. Reach for it when sizing what an uncapped surge costs you, never when quoting a per-minute price to a customer.
For an India comparison, Bolna starts near Rs 7 a minute and drops toward Rs 3 at volume, after a USD 6.3m seed round that General Catalyst led in January 2026.
Rs 7 to Rs 3 per minvendor claimBolna published pricing and funding announcement, January 2026, 2026 - The vendor published it. Worth carrying because minute economics swing enough between markets to change how hard a ceiling is worth defending.
Erlang formulas came out of traffic engineering for old telephone networks and are still where capacity planning for a telephony deployment starts.
-industry rangeErlang traffic engineering, standard telephony practice, 2026 - Roughly a century old and still earning its keep. Contact center planners drag it into voice AI rooms, and they are generally right to.
No industry benchmark has been published for peak-to-average ratios on AI voice deployments, which happens to be the one number a capacity plan is built on.
-no reliable figureYou measure your own or you guess. A ratio somebody quotes came off a different traffic pattern, and that says nothing about yours.
Provider ceilings move often and tend to be negotiated rather than posted, so any number written down here would be wrong shortly.
-no reliable figureGo to the provider documentation as it stands today, and ask what ceiling they will actually agree to rather than what the page advertises.
Examples
In practice
A retailer pushes a promotion to 200,000 subscribers at 10am. Four minutes later live sessions have gone past the concurrent stream cap at the upstream model provider. Nothing had been designed for that, so sessions break mid-conversation. The repair afterwards is a queue that states its wait, plus an automatic drop to text chat once concurrency crosses a threshold, which converts a broken experience into a worse one.
The everyday version
This is how many customers the system can talk to simultaneously. It only ever matters on one day: the day something you did landed and everybody showed up together. Which is the worst day available for it to fall over.
Usage
Who says it
- Infrastructure and platform engineers say it exactly, and with peak numbers attached.
- Procurement and vendor management say it as a capacity commitment they intend to sign.
- Contact center planners say it too, carrying real traffic modeling discipline over from telephony, and they are usually the most rigorous voice in the room.
Where it turns up
- Beside peak commitments, burst handling, rate limits, service level agreements, degradation behavior and how much notice a scale-up needs.
- The better RFPs ask what the system does once the limit arrives. The weaker ones ask only where the limit sits, which answers much less.
Common misuse
- Answering a capacity question with a monthly volume figure. The two measure different things.
- Sizing for average traffic, which books a failure for the exact moment traffic peaks.
- Leaving degradation undefined, which picks failure by default.
Questions people ask
What is concurrency in a voice AI system?
How many conversations run at once, counted as simultaneous active sessions. That is a separate measurement from total traffic: spread a thousand conversations over a day and nothing strains, land a hundred inside two minutes and you have an incident. It governs the busiest day, not the ordinary one.
What sets a voice agent's concurrency limit?
A provider quota, usually, rather than anything in your own code. Recognition, language and synthesis APIs impose requests per minute, tokens per minute or caps on concurrent streams, and that is often where the real ceiling sits. Trunk channels, audio processing and anything downstream such as a booking API cap it as well. Whichever runs out first is your number.
What happens when a voice agent hits its concurrency limit?
Four possibilities. It turns new sessions away cleanly, holds them in a queue with a stated wait, drops to a cheaper model or to text, or snaps mid-conversation. Snapping is what you get when nobody decided, and it is the worst of the four. The choice belongs in the product decisions before launch, and a good RFP asks to see it written down.
How many concurrent sessions should I provision for?
Size for the peak. Sizing for the average books a failure for the moment traffic actually arrives. No industry benchmark for peak-to-average ratios on AI voice deployments has been published, so that ratio has to come off your own traffic. For telephony, Erlang modeling is still where the analysis starts.
Last reviewed 30 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.