Time to first token
In one sentence
Time to first token (TTFT) is how long a language model takes to produce its first piece of output after a request, measuring the model alone rather than the full pipeline, which makes it one component of time to first audio and the most frequently misquoted latency number in voice AI.
Not to be confused with Time to first audio.
Definition
Time to first token measures how long a language model takes to produce its first piece of output after it receives a request.
It measures the model on its own, not the whole system, which is exactly why vendors reach for it and why it is so easy to misread.
TTFT is the model-level latency number, and the one that turns up most often in vendor comparisons, so knowing where its edges are is most of the point.
What the number actually contains
- Request transport to the inference endpoint.
- Queueing at the provider, which moves with load and stays largely invisible to you.
- Prefill, meaning reading the entire input prompt before a single token of output can be generated.
- Generating that first token.
Why prefill is the part that catches people out
- Prefill cost rises with input length. A long system prompt, a long conversation history and generous retrieved context each add to it.
- So TTFT is not a fixed property of a model. The same model returns a different TTFT depending on what you send it.
- A published figure is measured on one prompt size, usually a small one, while a production prompt that carries retrieval context is larger.
How it relates to time to first audio
- TTFT is one component of time to first audio. A voice pipeline wraps endpointing, transcription finalization, retrieval, synthesis and transport around it.
- A vendor who quotes TTFT as though it described conversational responsiveness is quoting about a third of the number that decides it.
- It is the most common latency overclaim in the category, and the easiest to check: ask where the clock starts and where it stops.
Why it still matters
- Model choice moves it a long way. The independently measured spread across leading realtime models is close to four times, which on its own can settle whether a product clears a one-second target.
- It is the component most directly under a team's control, through model choice and prompt discipline.
How to bring it down
- Trim the system prompt. The most available saving, and the one most often skipped.
- Retrieve fewer, better passages instead of more.
- Cache stable prompt prefixes where the provider supports it.
- Move to a faster model, accepting the quality tradeoff on purpose.
Common misconception
That TTFT is a fixed model specification. It moves with prompt size, provider load and geography. Read a published figure as a comparison under controlled conditions, not as a prediction of what your users will get.
Why it matters commercially
TTFT is the most quoted latency number in the category and the most often misapplied. The question worth asking a vendor is not what the TTFT is, but what the end-to-end time to first audio comes to on a realistic prompt, the kind that carries your retrieval context.
In voice specifically
In a text chat the first token can appear on screen the instant the model emits it, so TTFT is close to what the user feels. In a voice call the same token cannot be heard until it has been synthesized and played, so TTFT sits inside a larger budget and stops being the number the listener actually experiences.
Where AsqVox fits
AsqVox answers from documents you upload, so the passages it retrieves become part of the prompt the model reads before it can start. Fewer, better-chosen passages shorten that prompt, which lowers time to first token and sharpens the answer at the same time, one of the rare places where speed and quality pull the same way.
Visual
The number vendors quote, and the number people feel
Ask where the clock starts and stops. The answer is usually flattering.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
Measured time to first token across leading realtime models: around 0.78 seconds for xAI Grok Voice, 0.82 for OpenAI gpt-realtime-1.5, 1.14 for Amazon Nova 2 Sonic, and 2.98 for Gemini 3.1 Flash Live.
0.78s to 2.98sindependentArtificial Analysis, via Softcery, April 2026, 2026 - The fastest and slowest of the leading models are separated by close to four times, which is enough for the model choice on its own to settle whether a product comes in under a second. These are TTFT figures, so any complete pipeline sits above them.
Sub-second time to first audio is the working design target for conversational voice.
Under 1,000 msindustry rangeIndustry working threshold, 2026 - A working convention, not a published standard. TTFT is one segment inside this budget, not the whole of it.
Past roughly 1,500 milliseconds, a conversation is widely reported to feel broken.
1,500 msindustry rangeIndustry working threshold, 2026 - A shared working line rather than a measured cliff. A buyer feels it in a demo without being told to look for it.
The average gap between turns in human conversation is around 200 milliseconds, the baseline every voice design target is set against.
200 msindustry rangeWidely cited conversational turn-taking baseline, 2026 - A reference point rather than a target. It is quoted widely enough that no single study owns it, so cite it as the human baseline.
Production budgets commonly allow 150 to 300 milliseconds for the endpointing silence threshold, which runs before any model processing begins.
150 to 300 msindustry rangeCommon production budgeting convention, 2026 - An engineering convention rather than a research finding. It brackets TTFT on the input side of the pipeline.
Production budgets commonly allow 100 to 200 milliseconds for speech synthesis time to first byte, which runs after the model in the pipeline.
100 to 200 msindustry rangeCommon production budgeting convention, 2026 - Also an engineering convention, applying to streaming synthesis only. It brackets TTFT on the output side.
Published time to first token figures rarely state the prompt size they were measured on, which makes cross-provider comparison unreliable. There is no standard prompt configuration for TTFT benchmarking.
-no reliable figureThis is why two vendor TTFT numbers cannot be lined up side by side, and why a production prompt carrying retrieval context can post a figure well above the benchmark one.
Examples
In practice
A team measures TTFT at 700 milliseconds in isolation and is confident of a sub-second experience. Measured end to end, time to first audio comes back at 1,900 milliseconds, because endpointing, retrieval and synthesis contribute the rest. The model was never the constraint. The assumption that TTFT described the experience was.
The everyday version
Time to first token is how quickly the AI starts thinking out loud. It is the number you will see in comparisons, and it is not how long your customer waits, because there are several other steps on either side of it before any sound comes out.
Usage
Who says it
- AI engineers and platform teams, precisely, and usually with the model and prompt configuration attached.
- Vendors, in benchmark comparisons, frequently without stating the prompt size the figure was measured on.
Where it turns up
- Next to model performance, latency benchmarks and throughput on a spec sheet.
- A rigorous spec states the prompt configuration the number was measured under. Without it, two figures cannot be compared.
Common misuse
- Presenting TTFT as conversational responsiveness. The category-defining overclaim.
- Comparing figures across providers without matching prompt sizes.
- Treating it as a fixed model property rather than a load-and-input dependent one.
Questions people ask
What does time to first token measure?
It measures how long a language model takes to produce its first piece of output after it receives a request. That covers request transport, any queueing at the provider, prefill (reading the whole input prompt before generation can begin) and generating the first token. It measures the model alone, not the whole voice pipeline, which is why it is one component of time to first audio rather than the same thing.
What is the difference between TTFT and TTFA?
Time to first token measures the language model alone. Time to first audio measures the whole system, from the moment a person stops speaking to the moment sound comes back, so TTFT is one component of it. A vendor who quotes TTFT as conversational responsiveness is describing roughly a third of the number that decides it, which is the most common latency overclaim in voice AI.
Is time to first token a fixed number for a model?
No. Prefill cost rises with input length, so a long system prompt, a long conversation history or generous retrieved context all extend it, and the same model returns a different TTFT depending on what you send it. Provider load and geography move it as well. Read a published figure as a comparison under controlled conditions, not as a prediction of production performance.
How do you reduce time to first token?
Trim the system prompt, which is the most available saving and the one most often skipped. Retrieve fewer, better passages instead of more, since retrieval context adds to prefill. Cache stable prompt prefixes where the provider supports it. And choose a faster model, accepting the quality tradeoff on purpose, because the measured spread across leading models is close to four times.
Last reviewed 31 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.