Retrieval-augmented generation
In one sentence
Retrieval-augmented generation, or RAG, is a technique that makes an AI look the answer up in specific documents you have given it before it replies, instead of answering from whatever it absorbed during training.
Not to be confused with Context window, or Grounding.
Definition
RAG makes an AI answer out of documents you hand it, not out of whatever it happened to read during training. It looks the answer up first, then speaks.
Upload your price list and the price it quotes is your price. Change the file and the answer changes with it.
RAG separates knowledge from reasoning. Source documents are split into chunks, each chunk is turned into an embedding, meaning a numeric vector that stands for what the chunk says, and those vectors are stored in a vector database. When a question arrives it is embedded too, the closest chunks are pulled back by semantic similarity, optionally reordered by a reranking model, and passed to the language model inside the prompt as context. The model then writes an answer constrained to that context.
Why this beats fine-tuning for business knowledge
- Updating what the agent knows means re-uploading a document, not retraining a model.
- Retrieved passages can be cited, so an answer is auditable.
- A wrong answer usually traces back to a retrieval failure, which is fixable. Fine-tuning puts the fault in the model weights, where it is not.
Where RAG systems actually fail
- Chunking. Split a document badly and the answer lands across two chunks, neither of which gets retrieved.
- Retrieval, not generation. Most bad RAG answers are the model faithfully reporting the wrong passage.
- Conflicting sources. Two uploaded documents disagree and nothing arbitrates between them.
- Absence handling. Sometimes the correct answer is that the material does not cover it, and a system that cannot say so will invent something.
Telling true RAG from its impostors
- Pushing a whole document into the system prompt is not RAG. It is context stuffing, and it collapses as soon as the material outgrows the context window.
- Keyword search with a summarizing model on top is not RAG either. With no embeddings there is no semantic matching, so a question worded differently from the source text will miss.
Common misconception
That RAG eliminates hallucination. It cuts it down substantially and makes what is left traceable, but a model handed irrelevant retrieved context will still produce a confident wrong answer. Closing that gap is grounding, and it is a separate design problem.
Why it matters commercially
RAG is what lets a voice agent answer about your prices, your hours, your policies and your returns process. Take it away and you are left with something that speaks beautifully about a business it knows nothing about. That is worse than knowing nothing, because the mistakes come out fluent and they come out in front of customers.
In voice specifically
Spoken questions are longer, messier and more conversational than typed ones. A retrieval layer tuned on short keyword queries does badly on speech. Query rewriting, which turns a rambling spoken question into a clean retrieval query, matters more in voice than it does in text.
Where AsqVox fits
The Orb runs true retrieval over documents the business uploads. That is the mechanism behind a grounded answer rather than a plausible-sounding generic one.
Visual
How RAG answers a spoken question
The model is not remembering. It is looking it up.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
There is no single credible industry-wide accuracy figure for RAG. Accuracy depends entirely on corpus quality, chunking strategy and retrieval configuration.
-no reliable figureAny vendor quoting one number for RAG accuracy is quoting their own benchmark on their own data. Saying this plainly builds more trust than a borrowed percentage.
Whisper Large-v3 records around 2.7 percent word error rate on the clean LibriSpeech benchmark, but roughly 8 to 12 percent on real-world English audio.
2.7% against 8 to 12%independentWhisper Large-v3 published benchmark and reported real-world results, 2026 - Speech recognition error compounds retrieval error, so word error rate is a hard ceiling on answer accuracy in voice RAG. The distance between the benchmark and the real-world figure is the most useful fact on this page: it explains why a system that demos perfectly can disappoint in a noisy shop.
Deepgram Nova-3 reports around 5.26 to 6.84 percent word error rate in production conditions.
5.26 to 6.84%vendor claimDeepgram, Nova-3 reported figures, 2026 - Vendor-reported, so read it as a supplier best case. It still sits well above the clean benchmark number, which is the point.
An independent benchmark from Coval and Gradium found a speed against accuracy tradeoff across speech recognition providers, with no single leader on both axes.
No single leaderindependentCoval and Gradium speech recognition benchmark, 2026 - Worth citing because it contradicts every vendor claim of being the fastest and the most accurate at the same time.
Cost and latency both scale with the number of tokens processed, so putting a large corpus into every request is expensive and slow even where the context window technically allows it.
Scales with tokens processedindustry rangeEngineering convention across model providers, 2026 - Context windows have grown enormously, which leads some teams to conclude RAG is unnecessary. In voice, where sub-second response is the target, the token cost of stuffing everything into the prompt is disqualifying.
Examples
In practice
A logistics company uploads a 90-page tariff document, a returns policy and a service-area list. A visitor asks aloud what happens if they send something to a pin code the company does not cover. Two chunks come back: the exclusions from the service-area list, and the tariff clause that applies to them. The answer states the real policy, gives the surcharge, and names which of the three files it read. No part of it was invented, because no part of it had to be.
The everyday version
Your price list and your FAQ go in as files. A customer asks what something costs, and the figure they hear is the one written in your document, not the internet average for businesses that look like yours. Put your prices up, upload the new list, and the spoken answer moves with it.
Usage
Who says it
- Engineers and AI practitioners say RAG constantly and assume everyone in the room already understands chunking, embeddings and reranking.
- Buyers increasingly say it without knowing what it means, having been told it is the thing to ask for. That is an opening to explain it rather than nod along.
- Procurement cares about RAG mainly as a data-handling question: where the uploaded documents live and who can see them.
Where it turns up
- Next to knowledge base ingestion, supported file formats, refresh frequency, citation of sources, data residency and retention.
- The question that separates a serious RFP from a lazy one is what the system does when the corpus has no answer in it. A lazy one wants a yes or a no on whether RAG is supported.
Common misuse
- Marketing any document-aware feature as RAG. With no embeddings and no semantic retrieval, it is search with extra steps.
- Claiming RAG prevents hallucination outright. It constrains it and makes it traceable. Overclaiming here is the fastest way to lose a technical buyer.
Questions people ask
Does RAG stop an AI from making things up?
No, and a vendor who says it does is overclaiming. RAG cuts hallucination down substantially and makes what is left traceable, because you can see which passage the answer came from. A model handed irrelevant retrieved context will still answer confidently and wrongly, which is why grounding is treated as a separate design problem.
What is the difference between RAG and fine-tuning?
Fine-tuning changes the model weights. RAG leaves the model alone and changes what is put in front of it at question time. For business knowledge RAG usually wins, because updating means re-uploading a document instead of retraining, retrieved passages can be cited, and a wrong answer traces back to a retrieval failure you can fix.
Is putting a document into the prompt the same as RAG?
No. That is context stuffing, and it collapses as soon as the material outgrows the context window. Keyword search with a summarizing model on top is not RAG either, because with no embeddings there is no semantic matching, so a question worded differently from the source text will miss.
Why does my RAG system give wrong answers?
Almost always retrieval rather than generation, so the model is faithfully reporting the wrong passage. The usual causes are chunking that splits an answer across two chunks so neither is retrieved, two uploaded documents that disagree with nothing to arbitrate, and no way for the system to say that the material does not cover the question. In voice, speech recognition errors compound on top of all three.
Last reviewed 30 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.