Large language model
In one sentence
A large language model is a neural network trained on vast amounts of text to predict the next token, and that single capability at scale produces systems that answer questions, follow instructions and hold conversations.
Not to be confused with Small language model.
Definition
A large language model is the part of a voice system that decides what to say. It works by predicting what word comes next, over and over, very fast.
That is why it always sounds confident, including when it is wrong, and why grounding it in your own documents matters so much.
The mechanism is simpler than the behavior suggests. The model learns statistical patterns across enormous text corpora. Given a sequence, it predicts a probability distribution over what comes next, one option is selected, that selection is appended, and the process repeats token by token. Instruction-following behavior does not come from the raw pre-training. It comes from additional training stages afterward that shape the model toward helpful responses rather than plain continuation.
What follows from prediction-based generation
- Fluency is guaranteed. Accuracy is not. The model produces plausible text, and plausibility correlates with truth without being identical to it.
- Confidence in the output carries no information about correctness. This is the single most important practical fact about these systems.
- Knowledge is baked into the weights at training time and becomes stale, which is why retrieval exists.
What to weigh when picking one for voice
- Time to first token, because the visitor is waiting and can hear the silence.
- Instruction adherence, particularly the ability to keep answers short. Voice punishes verbosity far more than text does.
- Tool-calling reliability, if the agent takes actions.
- Cost per token, because voice conversations are multi-turn and costs accumulate.
- Context handling, since retrieval passages and conversation history compete for the same space.
- Raw capability on reasoning benchmarks is frequently less decisive than these practical properties.
The voice selection tension
- The most capable model is often not the fastest, and the independently measured spread in time to first token across leading realtime models is nearly four-fold.
- A slightly less capable model that responds in 800 milliseconds usually produces a better voice product than a stronger one taking 2,500 milliseconds.
Common misconception
That a larger or more capable model produces a better voice agent. Beyond a threshold of competence, latency, brevity and grounding matter more. Voice rewards different properties from text.
Why it matters commercially
Model selection is a recurring decision with direct effects on cost, latency and quality. It is also the component most likely to be changed, since providers release new versions frequently, so whether the buyer can choose or swap the model is a question worth asking before signing.
In voice specifically
In text a wordy, thorough answer reads as a strength and a 2,500 millisecond response is barely noticed. In voice both are penalties. The visitor hears every second of latency as dead air and every extra sentence as a wait, so a model that answers fast and keeps it short beats a stronger, slower, chattier one at the same task.
Where AsqVox fits
The model is one block of five in the AsqVox voice pipeline. Retrieval quality and turn-taking determine more of the experience than model choice does, provided the model clears a competence and latency threshold.
Visual
What the model actually does
Predict, select, append, repeat. That is the whole mechanism.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
Time to first token across leading realtime models spans nearly four-fold: xAI Grok Voice around 0.78 seconds, OpenAI gpt-realtime-1.5 around 0.82, Amazon Nova 2 Sonic around 1.14, and Gemini 3.1 Flash Live around 2.98.
0.78s to 2.98sindependentArtificial Analysis via Softcery, 2026 - The near four-fold spread means model selection alone can determine whether a product clears a one-second target. These figures move with every release, so date and re-check them.
Sub-second time to first audio is the working target for conversational voice, and above roughly 1,500 milliseconds conversations feel broken.
sub-second target, 1,500ms ceilingindustry rangeEngineering convention for conversational voice, 2026 - The model is only one contributor to time to first audio, but a slow model can spend the entire budget on its own.
All-in production voice cost lands around USD 0.11 to USD 0.33 per minute, of which model inference is one component.
USD 0.11 to 0.33 per minuteindustry rangeIndustry-reported production voice cost range, 2026 - Model inference is one line in a multi-line cost, which is why treating the model as the whole cost picture misleads.
Model capability benchmarks change with every release and any specific figure ages within months. Point readers at current provider documentation and independent leaderboards rather than publishing a number.
-no reliable figureCiting a specific benchmark score in durable documentation dates the document. Direct to a live leaderboard instead.
There is no published benchmark measuring instruction adherence for brevity specifically, despite this being one of the most decisive properties for a voice agent.
-no reliable figureThe property that most affects a voice product, keeping answers short on command, is the one nobody publishes a number for. Test it on your own prompts.
Examples
In practice
A team upgrades to a more capable model and user satisfaction falls. The new model produces longer, more thorough answers, which read well and take substantially longer to hear. Constraining response length restores satisfaction. The model was better at the task and worse at the medium.
The everyday version
A large language model is the part of a voice agent that decides what to say. It works by predicting what word comes next, over and over, very fast. That is why it always sounds confident, including when it is wrong, and why grounding it in your own documents matters so much.
Usage
Who says it
- Everyone, across engineering, business and media, frequently without precision.
- Engineers specify which model and which version, because the differences between them are large.
Where it turns up
- In an RFP, next to model selection, provider, versioning policy, how the model provider handles data and fallback behavior on a provider outage.
- Whether the buyer can choose or change the model is a question worth asking, since it is the component most likely to be swapped.
Common misuse
- Assuming greater capability produces a better voice agent.
- Treating the model as the product. It is one component of several.
- Citing specific benchmark figures in durable documentation, since they age quickly.
Questions people ask
Does a more capable language model make a better voice agent?
Not beyond a threshold of competence. Once a model is good enough, latency, brevity and grounding matter more than raw capability. A model that answers in 800 milliseconds usually produces a better voice product than a stronger one taking 2,500 milliseconds, because the visitor hears the difference as silence.
How does a large language model actually work?
It predicts a probability distribution over what token comes next, selects one, appends it, and repeats, token by token. That is the whole mechanism. The ability to follow instructions rather than just continue text comes from training stages that run after the base pre-training.
Why does an LLM sound confident even when it is wrong?
Because it generates plausible text, and plausibility correlates with truth without being the same thing. Confidence in the output carries no information about whether it is correct. That is the single most important practical fact about these systems, and it is why grounding answers in your own documents matters.
What should I look at when choosing a model for voice?
Time to first token, instruction adherence for keeping answers short, tool-calling reliability if the agent acts, cost per token across multi-turn conversations, and context handling. Reasoning benchmark scores are less decisive than most buyers assume. Ask whether you can change the model later, because you probably will.
Last reviewed 3 August 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.