Diarization
In one sentence
Diarization is the process of working out who spoke when in a recording, separating a conversation into anonymous speaker turns without needing to know the identity of any speaker.
Not to be confused with Speaker identification.
Definition
Diarization works out who spoke when in a recording. It separates a conversation into speaker turns and labels them speaker one, speaker two and so on.
It does this without knowing, or needing to know, the identity of any speaker.
Diarization answers one narrow question, who spoke when. That is distinct from recognition, which answers what was said, and from speaker identification, which answers who a person is. Keeping the three apart matters, because they carry very different legal weight.
How it works
- The audio is cut at the points where the speaker probably changes.
- Each segment is turned into a speaker embedding, a numeric fingerprint of the voice.
- Segments are clustered so that pieces from the same voice group together.
- Each cluster gets an anonymous label such as speaker one and speaker two.
- The system never needs anyone's identity for any of this to work.
Where it gets hard
- Overlapping speech. When two people talk at once, segmentation degrades badly, and this is the dominant source of error in real conversations.
- Similar voices. Two speakers with close vocal characteristics get clustered together as one.
- Short turns. Backchannel sounds and one-word interjections are too brief to embed reliably.
- Unknown speaker count. A system that has to infer how many people are present does worse than one told the number in advance.
- Poor audio. A bad channel degrades the speaker embeddings before clustering even begins.
Where it earns its place
- Meeting and call transcription, where attributing each statement to a person is the whole point.
- Contact center analytics, separating agent from customer, which is usually easy because the two sit on separate audio channels.
- Multi-party situations such as a speakerphone in a room full of people.
- Preparing training data.
The channel shortcut
- When each speaker is already on a separate audio channel, which is common in telephony, splitting them is trivial and diarization is not needed at all.
- Single-channel audio with everyone mixed together is the case that actually needs diarization, and it is a great deal harder.
The metric
- Diarization error rate rolls missed speech, false alarm speech and speaker confusion into one percentage.
- Like word error rate, it treats every error as equal, regardless of how much any single one mattered.
Common misconception
That diarization identifies speakers. It groups them. Putting a name to a group requires either speaker identification, which is a biometric operation with its own legal implications, or working out who is who from what was actually said.
Why it matters commercially
For a website voice agent it barely applies, because there is one visitor per session and nobody else to tell apart. It earns its keep in transcription, meeting tools and multi-party audio, and it is worth understanding as the edge of what a system knows about who is speaking, as opposed to what was said or who the person is.
In voice specifically
A text chat never needs this. Every typed message already carries the identity of whoever sent it, so there is nothing to separate. Diarization is a problem that exists only because audio can arrive as a single undifferentiated stream with several people mixed into it.
Where AsqVox fits
Not applicable in the usual case. A conversation with the Orb on a website is single-speaker by nature, one visitor talking to one agent, which removes a whole class of difficulty that telephony and meeting tools have to handle.
Visual
Who spoke when, without knowing who they are
The labels are anonymous. Diarization groups voices, it does not put names to them.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
Diarization error rate is the standard score, and it folds three separate mistakes into one percentage: missed speech, false alarm speech and speaker confusion.
One percentage figureindustry rangeStandard diarization evaluation practice, 2026 - Like word error rate, it weights every error equally, so a percentage on its own does not tell you whether the failures landed on the turns that mattered.
Under GDPR, biometric data processed to uniquely identify a person is special category data under Article 9. Diarization that only groups voices anonymously does not engage this, while speaker identification does.
GDPR Article 9industry rangeEU General Data Protection Regulation, Article 9, 2026 - A legal distinction rather than a measurement, and a genuinely meaningful one. Anonymous grouping and named identification sit in different legal categories, so do not describe a diarization feature as if it identified anyone.
No single authoritative benchmark measures diarization performance under realistic multi-party conditions in a way that lets providers be compared cleanly. Published figures use different datasets and different assumptions about how many speakers are present.
-no reliable figureAsk any vendor quoting a diarization number for the dataset, the assumed number of speakers, and how overlap was handled, because those choices move the figure more than the model does.
Overlapping speech is widely accepted as the dominant error source in real conversations, but there is no standard published figure for how much it degrades performance.
-no reliable figureThe direction is not in dispute. The size of the effect is not established, so treat any single overlap-degradation percentage as illustrative.
Examples
In practice
A meeting transcription tool does well on orderly calls where people take turns and badly on loose discussions where everyone talks over each other. Scoring diarization error rate separately for the overlapped and non-overlapped stretches pins the blame on overlap. The workable fix is not a stronger model, it is capturing each speaker on a separate channel wherever that is possible.
The everyday version
Diarization is the job of working out which parts of a recording came from which person, without knowing who those people are. It tags them speaker one and speaker two. Attaching real names is a separate job, and a much more legally sensitive one.
Usage
Who says it
- Speech engineers and transcription product teams.
- Contact center analytics teams, though separate audio channels often make it unnecessary for them.
Where it turns up
- On a spec sheet it sits next to transcription features, speaker labeling, multi-channel support and meeting integrations.
- It is the feature that tells you whether a transcript can attribute each line to a person, or only render the words.
Common misuse
- Confusing diarization with speaker identification, which carries different and much heavier legal exposure.
- Expecting reliable results on heavily overlapped, informal conversation.
- Reaching for it where separate channels are already available and would do the job trivially.
Questions people ask
What is the difference between diarization and speaker identification?
Diarization groups speech by voice without any names attached, answering who spoke when. Speaker identification puts a name to a voice by matching it against known voiceprints, answering who this is. Identification is a biometric operation with heavier legal requirements, while diarization that only groups voices anonymously is not.
What is diarization error rate?
It is the standard score for diarization, and it combines three kinds of mistake into a single percentage: missed speech, false alarm speech and speaker confusion. Like word error rate, it weights every error equally, so a low number does not by itself tell you whether the failures fell on the turns that mattered.
Why does diarization struggle with overlapping speech?
When two people talk at once, the step that segments the audio at speaker changes degrades badly, and overlap is widely accepted as the dominant error source in real conversations. There is no standard published figure for how much it degrades performance. The practical fix is not a stronger model but capturing each speaker on a separate channel where possible.
Does a website voice agent need diarization?
Usually not. A website conversation is one visitor talking to one agent, so there is only a single speaker and nobody else to tell apart. Diarization matters for meeting tools, call transcription and multi-party audio, where attributing each statement to a person is the point.
Last reviewed 31 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.