Speech technology

Automatic speech recognition

Automatic speech recognition (ASR), also called speech to text, is the technology that turns spoken audio into written text, and it is the first real step in almost every voice agent.

Barge-in

Barge-in is a voice agent stopping its own speech the instant a person starts talking over it, which is what makes the exchange feel like a conversation rather than a recording you have to sit through.

Endpointing

Endpointing is how a voice agent decides that the person has finished speaking and its own turn has begun, normally by timing the silence against a threshold set between 150 and 300 milliseconds.

Speech-to-speech models

A speech-to-speech model is a single model that consumes audio and emits audio directly, with no transcription step in the middle, which buys latency and preserved tone at the cost of the text record a cascaded pipeline produces as a by-product.

Text to speech

Text to speech (TTS), also called speech synthesis, turns written text into spoken audio, and it is how a voice agent gets its voice.

Turn detection

Turn detection is working out whose turn it is to speak, and semantic turn detection does it by judging whether the sentence sounds finished rather than by timing the silence after it.

Voice activity detection

Voice activity detection (VAD) is the component at the very front of a voice agent that decides, frame by frame, whether the incoming audio contains speech or only background noise.

Voice cloning

Voice cloning builds a synthetic copy of one named person's voice out of a recording of them talking, and that copy can then be made to say things the person never said.