Realtime API
In one sentence
A realtime API is an interface built for live, low-latency, bidirectional audio streaming rather than request-and-response calls, and it is what makes conversational voice possible without stitching separate recognition, reasoning and synthesis services together.
Definition
A realtime API is an interface built for live, low-latency, bidirectional audio streaming rather than request-and-response calls.
It is what makes conversational voice work without wiring separate recognition, reasoning and synthesis services together by hand.
A conventional API follows one pattern: send a complete input, get a complete output back. Conversation does not work like that, and realtime APIs exist to close the gap.
What sets them apart
- A persistent connection, usually WebSocket or WebRTC, held open for the whole session instead of opened for each request.
- Bidirectional streaming, so audio flows in and out at the same time and continuously.
- An event-driven interface. It emits events, such as speech detected, transcript updated, response started and response interrupted, rather than returning one result.
- Server-side turn management. Many realtime APIs run voice activity detection and turn-taking that the application would otherwise have to build.
- Interruption handling built in, so the client can signal a cancellation and the server stops generating.
What it replaces
- The alternative is orchestrating separate recognition, language model and synthesis services, managing the streaming between them, and implementing turn-taking in your own code.
- That route gives you more control and more work. It also lets you mix best-of-breed components instead of taking one provider's whole stack.
The architectural tradeoff
- Realtime APIs, speech-to-speech ones in particular, cut both latency and implementation burden.
- They also cut inspectability. A cascaded architecture has a text bottleneck you can log, filter and ground against. A native audio path has none by default.
- Retrieval injection and guardrail enforcement are harder without that text layer, which is why hybrid architectures exist.
Practical concerns
- Connection establishment. Opening a session takes time, and that time lands on first-word latency unless the connection is pre-warmed.
- Reconnection. Networks drop, and a session lost mid-conversation has to be recovered without losing state.
- Cost model. Audio token pricing is not the same as text pricing and is not directly comparable across providers, because tokenization schemes differ.
- Rate limits and concurrency caps apply, and they are often the binding constraint on how much a deployment can handle.
Common misconception
That a realtime API removes the need to think about latency. It removes several sources of latency and adds one of its own, connection establishment in particular, so the end-to-end budget still has to be measured.
Why it matters commercially
Realtime APIs lowered the barrier to building conversational voice a long way, which is much of why the category grew so fast. They also concentrate dependency on one provider, which is a strategic question rather than a technical one, and it belongs in a procurement conversation.
In voice specifically
A text chat needs no held-open audio line. Messages go back and forth as discrete requests, which is exactly the request-and-response shape a realtime API exists to escape. The persistent connection is a cost that only voice pays.
Where AsqVox fits
This is underlying infrastructure. For a product grounded in a customer's own uploaded documents, the question a realtime API raises is how retrieval and audit logging work when the conversational path is native audio, which is the hybrid pattern covered under speech-to-speech models.
Visual
Request and response, against a held-open line
A realtime API removes several sources of latency and adds one of its own. Measure the whole budget.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
Measured time to first token across leading realtime models: around 0.78 seconds for xAI Grok Voice, 0.82 for OpenAI gpt-realtime-1.5, 1.14 for Amazon Nova 2 Sonic, and 2.98 for Gemini 3.1 Flash Live.
0.78s to 2.98sindependentArtificial Analysis, via Softcery, April 2026, 2026 - The clock here stops at the model's first token rather than at audible speech, so a deployed pipeline sits above these figures. With the fastest and slowest almost four times apart, model choice can settle the sub-second question on its own.
Sub-second time to first audio is the working target these measurements are read against.
Under 1,000 msindustry rangeIndustry working threshold, 2026 - A working convention rather than a published standard. No body sets it and no vendor is audited against it.
Past roughly 1,500 milliseconds, an exchange starts to feel broken.
1,500 msindustry rangeIndustry working threshold, 2026 - Same status as the sub-second target: a shared working line rather than a measured cliff.
The human conversational turn gap averages around 200 milliseconds.
200 msindustry rangeWidely cited conversational turn-taking baseline, 2026 - A reference point, not a target. Cite it as the human baseline rather than as a measurement of anything you built.
No published benchmark measures realtime API connection establishment time, which varies with network conditions and geography.
-no reliable figureIt is the source of latency the API adds rather than removes, and the one a deployment has to measure for itself on representative networks.
Audio token pricing is not directly comparable across providers, because tokenization differs, so cost comparison needs modeling on representative traffic rather than a reading of rate cards.
-no reliable figureTwo providers can quote per-token prices that do not describe the same unit. State the absence rather than lining the numbers up.
Provider rate limits and concurrency caps change frequently and are often negotiable rather than published.
-no reliable figureThey are also frequently the binding constraint on capacity, so treat a published cap as a starting point for a conversation rather than a fixed ceiling.
Examples
In practice
A team moves off an orchestrated pipeline onto a realtime API and takes several hundred milliseconds off its median response time. Then it finds first-interaction latency has got worse, because opening the connection now happens on the click instead of being paid for in advance. Pre-warming the connection on page load, before the visitor does anything, gives the time back.
The everyday version
A realtime API is a phone line left open, not a stack of letters sent back and forth. It is the technical reason voice agents got good enough to be worth deploying over the last couple of years. It also means the company that provides it sits fairly close to the center of how your product runs.
Usage
Who says it
- AI and platform engineers, routinely.
- It appears in provider documentation and architecture decision records.
- It is rarely a buyer term, though provider dependency is a legitimate procurement question.
Where it turns up
- On a spec sheet it sits beside the transport protocol, the supported models, concurrency limits, rate limits, interruption handling and reconnection behavior.
Common misuse
- Assuming it eliminates latency concerns.
- Comparing audio token pricing across providers as though the units match.
- Overlooking provider concentration risk when the whole realtime path runs on a single vendor.
Questions people ask
What is a realtime API?
A realtime API is an interface built for live, low-latency, bidirectional audio streaming rather than request-and-response calls. It holds a single connection open for the whole session, streams audio in and out at once, emits events such as speech detected and response interrupted, and usually handles turn-taking and interruption on the server. It is what makes conversational voice possible without stitching separate services together.
What is the difference between a realtime API and a conventional API?
A conventional API is request-and-response: you send a complete input and receive a complete output, and it cannot start replying until the whole input has arrived. A realtime API keeps a persistent connection open, streams audio in both directions continuously, and is event-driven. The realtime model fits conversation; the request-and-response model does not.
Does a realtime API remove the need to worry about latency?
No. It removes several sources of latency, mainly by dropping handoffs between services, but it adds one of its own in connection establishment, which lands on first-word latency unless the session is pre-warmed. The end-to-end budget still has to be measured. Independent time-to-first-token figures for the models themselves ranged from around 0.78 to 2.98 seconds in April 2026, and a deployed pipeline lands higher.
How fast are realtime voice APIs?
Measurements from Artificial Analysis via Softcery in April 2026 put time to first token at around 0.78 seconds for xAI Grok Voice, 0.82 for OpenAI gpt-realtime-1.5, 1.14 for Amazon Nova 2 Sonic and 2.98 for Gemini 3.1 Flash Live. Those are model-level figures, so a deployed pipeline including connection, transport and playback lands higher, and the roughly fourfold spread means model choice alone can settle whether you come in under a second.
Last reviewed 31 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.