Vector database
In one sentence
A vector database holds embeddings and returns the nearest ones to a query fast enough that nobody waits, which makes it the search index that meaning-based retrieval runs on.
Not to be confused with Embeddings.
Definition
A vector database keeps embeddings and pulls back the ones nearest to a question, quickly enough that nobody notices the wait.
It is the search index for meaning-based retrieval, in the same way a book index is the search index for a book.
Everything it does reduces to one operation: find the nearest neighbors of a query vector inside a large collection of high-dimensional ones. Doing that exactly gets impossibly slow once the collection is big, so anything running in production reaches for approximate methods, which surrender a little recall and buy back an enormous amount of speed.
The indexing approaches worth knowing by name
- HNSW, hierarchical navigable small world graphs. Constructs proximity edges in tiers. Quick, recall stays high, appetite for memory is large. Where response time is the binding constraint, this is what most teams pick.
- IVF, inverted file indexing. Carves the space into clusters and only looks inside the ones sitting closest to the query. Cheaper on memory, at the price of tuning how many clusters get probed.
- Product quantization. Compresses vectors to cut memory at some cost to accuracy. Frequently combined with either of the above.
The recall and latency tradeoff
- Sometimes the genuinely closest vector is simply not returned. How often that happens is a setting, not an accident.
- It is invisible in testing and shows up in production as occasional unexplained wrong answers. Worth knowing, because a retrieval failure can be an index configuration problem rather than an embedding or chunking one.
Features that matter more than raw search speed
- Metadata filtering. Constraining a lookup by document type, date, customer or permission level. Non-negotiable once several businesses share one deployment, since one customer's documents must never appear in another customer's answers.
- Hybrid search. Combining vector and keyword scoring inside a single query.
- Incremental updates. Documents can go in and come out without the whole index being rebuilt, which is the difference between a new price list going live in seconds and going live overnight.
- Namespace or tenant isolation. Read this as a security control first and a feature second.
The build-or-buy question, which is smaller than it looks
- Dedicated vector databases, vector extensions to an existing database, and embedded libraries all exist and all work. For most voice agent workloads the corpus is modest, so the choice comes down to operational preference rather than to scale.
- A typical business deployment holds a few hundred to a few thousand chunks, which is small by vector database standards. The engineering discussion is often considerably more elaborate than the workload requires.
Common misconception
That picking the vector database is the hard part of RAG. For the corpus sizes a business voice deployment actually carries, it is the part that gives least trouble. The difficulty, and the quality still left on the table, sits in how documents get split and how retrieval gets evaluated.
Why it matters commercially
Two properties carry commercial weight. Latency, because retrieval sits directly inside the time-to-first-audio budget and every millisecond it takes is a millisecond of silence the visitor hears. And isolation, because leakage between customers' documents in a multi-tenant deployment is an incident, not a defect.
In voice specifically
In a text interface a slow lookup is a spinner. In voice it is silence, and silence is the one thing a listener cannot interpret. A retrieval step that takes several hundred milliseconds does not read as a slow database to the person waiting. It reads as an agent that did not hear the question.
Where AsqVox fits
Storing and searching vectors is what the Orb's true RAG over uploaded documents runs on. Where several customers share one deployment, isolation between them is the property that counts, and how long a lookup takes is heard directly as how responsive the widget feels to whoever is waiting.
Visual
The index that has to answer in milliseconds
At business corpus sizes, this is the easy part. Chunking is not.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
There is no one performance figure for vector databases that survives a change of workload. Every published benchmark is tied to a particular dataset, dimensionality, index configuration and recall target.
-no reliable figureVendors select the combination that flatters them, which is not dishonest so much as unavoidable. Treat every comparative claim as configuration-dependent, and benchmark on your own corpus at your own recall target if the choice matters.
Giving up a little recall to gain speed is the design intent of approximate nearest-neighbor search, not a compromise. Exact search cannot hold a production latency target once the corpus reaches any real size.
Recall traded for speedindustry rangeAlgorithmic property of approximate nearest-neighbor search, 2026 - An algorithmic fact rather than a vendor characteristic. Nobody is cutting a corner by using approximate search. Everybody is, and the only question is where the configuration sits.
Production vector search is dominated by two index families, HNSW and IVF, with product quantization layered over either one when memory needs cutting.
HNSW and IVFindustry rangeProduction vector search convention, 2026 - A description of what is deployed rather than a ranking of what is best. HNSW is the more common choice where latency is the binding constraint.
Conversational voice is designed against a sub-second time to first audio target, and the retrieval step spends part of that budget.
Under 1,000 msindustry rangeIndustry working threshold, 2026 - A working convention rather than a published standard. It is the reason index configuration is a product decision and not only an infrastructure one.
Above roughly 1,500 milliseconds in total, conversations are widely reported to feel broken.
1,500 msindustry rangeIndustry working threshold, 2026 - A retrieval step consuming hundreds of milliseconds is taking a large fraction of the whole budget, which is precisely why index configuration has a direct product consequence.
Examples
In practice
Over several months, with not one line of code changed, a deployment slides from 1,100 milliseconds to first audio to 1,900. Stage-by-stage timing puts the blame on retrieval: 90 milliseconds at launch, 800 now, after a corpus that quadrupled while no index was ever built over it. An HNSW index drops it back under 100 milliseconds. Every symptom along the way looked like model latency, and the model had nothing to do with it.
The everyday version
Picture a filing cabinet that can put its hand on the right page of your documents in a fraction of a second. That speed is why your customer hears the answer while they are still paying attention, instead of some time after they stopped.
Usage
Who says it
- Engineers and data platform teams, routinely.
- Almost never reaches a buyer, and there is no reason it should.
- Security teams, whose interest is narrow: how tenants are kept apart, and whether data is encrypted at rest.
Where it turns up
- Usually in the same clause as encryption at rest, data storage location, tenant isolation, retrieval latency, and retention and deletion.
- Deletion is the clause worth reading twice. Removing a document has to remove its vectors, and some architectures make that harder than anyone expected at signing.
Common misuse
- Building for a scale that never turns up. Business voice corpora are usually tiny.
- Treating the choice of vector database as the thing that determines RAG quality. Chunking and retrieval evaluation matter considerably more.
- Taking it on trust that deleting a source document makes it unretrievable. Whether that holds is an implementation detail, so check rather than assume.
Questions people ask
What does a vector database actually do?
It stores embeddings and answers nearest-neighbor queries across a large set of high-dimensional vectors. Exact nearest-neighbor search is far too slow at scale, so production systems use approximate algorithms that give up a small amount of recall in exchange for very large gains in speed.
What is the difference between HNSW and IVF?
HNSW builds a layered proximity graph and is fast with high recall, at the cost of memory, which makes it the usual choice for latency-sensitive work. IVF partitions the space into clusters and searches only the nearest ones, using less memory but needing tuning of how many partitions to probe. Product quantization compresses vectors to save memory at some cost to accuracy and is frequently combined with either.
Do I need a dedicated vector database?
Often not. A typical business voice deployment indexes a few hundred to a few thousand chunks, which is small by vector database standards. Dedicated databases, vector extensions to an existing database, and embedded libraries all handle that comfortably, so the decision usually comes down to operational preference rather than scale.
Why does retrieval go wrong in production but not in testing?
Approximate search occasionally misses the true nearest neighbor, and the configuration decides how often. It surfaces as an occasional unexplained wrong answer rather than as a visible failure, which is why a retrieval problem can turn out to be an index configuration issue rather than an embedding or chunking one.
Last reviewed 30 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.