LLMs, RAG and orchestration

Chunking

Chunking is splitting documents into smaller passages before they are indexed for retrieval, and where those splits fall largely decides whether the right answer can be found at all.

Context window

The context window is how much text a model can consider at once, counting both what it has been given and what it produces, and everything outside the window does not exist as far as the model is concerned.

Conversation memory

Session state is what a voice agent remembers within a single conversation and conversation memory is what it remembers across conversations, and the two are different problems carrying different risks.

Embeddings

An embedding is a list of numbers representing the meaning of a piece of text, arranged so that two passages about the same thing sit close together even when they share no words, which is what lets a system search by meaning instead of by keyword.

Function calling

Function calling is how a language model acts rather than only speaks: it emits a structured request for an action, and the application around it checks that request, runs it against a real system, and feeds the result back into the conversation.

Grounding

Grounding is the discipline of tying what an AI says to a verifiable source, so the answer can be traced to real material rather than to whatever the model absorbed during training.

Guardrails

Guardrails are the limits that keep a voice agent from saying or doing what it should not, whatever the visitor asked and whatever the system retrieved, and the thing that makes them guardrails is that they hold outside the model instead of being asked of it.

Hallucination

A hallucination is when an AI states something false with the same confidence it states something true, which is not a lie, because there is no intent, and not a bug, because the system is working as designed.

Human handoff

Human handoff is the act of moving a live conversation from an automated agent to a person, and escalation is the judgement that the move is needed, and together they are what a customer runs into whenever the automation could not finish the job.

Reranking

Reranking is a second pass that reorders retrieved results by relevance using a slower and more accurate model than the initial search, and it is what fixes the near-misses fast retrieval gets wrong.

Retrieval-augmented generation

Retrieval-augmented generation, or RAG, is a technique that makes an AI look the answer up in specific documents you have given it before it replies, instead of answering from whatever it absorbed during training.

Vector database

A vector database holds embeddings and returns the nearest ones to a query fast enough that nobody waits, which makes it the search index that meaning-based retrieval runs on.