Emotion detection and affective computing
In one sentence
Emotion detection attempts to infer a person's emotional state from signals such as their voice, and affective computing is the broader field of systems that recognize, interpret or simulate emotion; the acoustic measurement is real, but the inference from it to a named emotion is scientifically contested.
Not to be confused with Sentiment analysis.
Definition
Emotion detection tries to work out how someone feels from signals such as the sound of their voice. Affective computing is the wider field of systems that recognize, interpret or simulate human emotion.
This is the most scientifically contested area in the voice category, and any page that does not say so is misleading its reader.
What is actually being measured
- Acoustic features: pitch, energy, speaking rate, spectral characteristics, voice quality.
- Linguistic content, meaning what is actually said. It is a separate signal, and it often gets bundled into the same product.
- These are real, measurable properties. The contested step is the jump from those properties to a discrete named emotion.
Why the inference is disputed
- The mapping from vocal characteristics to emotional states is not consistent across individuals, cultures or contexts. The same acoustic pattern can mean different things in different people.
- Discrete emotion categories, treating emotion as a small set of labeled buckets, are themselves contested within psychology as a description of how emotion works.
- Systems trained mostly on acted emotional speech, which is common because it is easy to collect, learn to recognize performed emotion rather than felt emotion. The two are measurably different.
- Demographic performance disparities have been found repeatedly across affective computing systems, so accuracy varies by group in ways that create fairness problems.
- The criticism is substantial and long-standing, and it lands on the inference layer, not the measurement layer.
The regulatory position, which follows the science
- The EU AI Act prohibits emotion inference in workplace and educational settings under its prohibited practices provisions, applied from 2 February 2025. It is an absolute prohibition in those contexts, not a compliance-with-conditions category.
- The prohibition exists partly because of the contested reliability of the inference. Regulators reached a firmer conclusion than the market has.
- Other contexts are not prohibited, but a ban in employment and education is a strong signal about how the technology is viewed.
The defensible version of the capability
- Measuring the acoustic properties and describing them, rather than labeling emotions. "Speaking rate rose and pitch went up" is a measurement. "The customer is angry" is an inference.
- Using the signals as triage indicators rather than conclusions, for instance flagging a conversation for review instead of making a decision about the person.
- Being explicit that the output is a signal with limited reliability, not a reading of someone's inner state.
Common misconception
That voice emotion detection reliably tells you how someone feels. The measurement is real; the inference is contested. A system that reports a customer as angry has made a probabilistic guess from acoustic features, and treating that guess as a fact about a person is where the harm starts.
Why it matters commercially
Emotion detection is heavily marketed and carries both scientific and regulatory risk. Using it in employment or educational settings in the EU is prohibited outright. Using it elsewhere as the basis for decisions about individuals is a fairness and accuracy exposure. Using it as a triage signal with honest caveats is defensible.
In voice specifically
Text carries only the words. Voice adds pitch, energy and pace, which is why emotion detection is pitched as a voice feature in the first place. That extra channel is also the contested one: the words can be classified as an observable artifact, but the sound of them only supports a guess about the speaker's state.
Where AsqVox fits
The Orb's dashboard includes sentiment, which is described carefully and kept distinct from emotion inference. Sentiment analysis of what was said is a different and better-established operation, and that distinction is worth holding rigorously rather than blurring for marketing convenience.
Visual
What is measured, and where the dispute begins
The measurement is real. The label is a guess about someone's inner state.
Statistics
Every figure carries its source and year. Vendor numbers are labelled as vendor numbers, and where no reliable figure exists this page says so rather than borrowing one.
The EU AI Act prohibits emotion inference in workplace and educational institutions under its prohibited practices provisions, applicable from 2 February 2025.
Prohibited from 2 Feb 2025industry rangeEU AI Act, prohibited practices provisions, 2025 - A regulatory fact and one of the firmest citations available on this topic. It is an absolute prohibition in those settings, not a compliance-with-conditions category.
EU AI Act Article 50 transparency obligations become enforceable on 2 August 2026.
Enforceable 2 Aug 2026industry rangeEU AI Act, Article 50, 2026 - Distinct from the workplace and education prohibition above. Cite the two dates separately, because they cover different obligations.
There is no reliable, independently verified accuracy figure for voice-based emotion detection that transfers across populations and contexts. Vendor accuracy figures are measured on their own datasets, frequently containing acted rather than spontaneous emotion.
-no reliable figureState this prominently. On this topic the absence of a transferable accuracy number is the single most important thing a reader can know.
There is no standard benchmark that permits comparison across emotion detection providers.
-no reliable figureWithout a shared benchmark, two vendor accuracy claims cannot be lined up against each other, and neither predicts performance on your own calls.
Demographic performance disparities have been identified across affective computing research, but there is no single authoritative published quantification for voice emotion detection specifically.
-no reliable figureThe disparity is documented as a direction; the magnitude for voice emotion detection is not settled. Report the existence of the problem, not a borrowed figure for its size.
Real-world speech recognition word error rates run at roughly 8 to 12 percent, which means any system combining acoustic and linguistic signals is working from imperfect text on the linguistic side.
8% to 12%industry rangeReported real-world English word error rate, Whisper Large-v3, 2026 - A reported band rather than a single measurement. It matters here because the linguistic half of an emotion detection product inherits every transcription error before it starts.
Examples
In practice
A contact center deploys voice emotion detection to score agent calls. When the results are analyzed, scores differ systematically by speaker accent, because the acoustic patterns the model reads as agitation overlap with normal speech in some speaker groups. The system is producing a demographic artifact and presenting it as an emotional assessment. It is pulled from evaluative use and kept only as a flag for human review.
The everyday version
Emotion detection tries to work out how someone feels from how they sound. It can genuinely measure things like whether a person is speaking faster or louder. Whether that means they are angry is a guess, and it is a guess that works better for some people than others. In Europe, using it to assess employees or students is now banned outright.
Usage
Who says it
- Vendors, enthusiastically, and often without the scientific caveats.
- Researchers in affective computing, with a good deal more nuance.
- Regulators and ethicists, critically.
- Contact center buyers, who meet it as a feature and rarely question the inference.
Where it turns up
- Next to conversation analytics, agent performance, quality management and customer-experience scoring.
- In the EU it belongs next to prohibited-practices screening, because the workplace ban applies directly to agent-performance use cases.
Common misuse
- Presenting inferred emotion as measured fact.
- Deploying it in employment or educational contexts within the EU, which is prohibited.
- Using it as the basis for a decision about a person rather than as a triage signal.
- Conflating it with sentiment analysis of content, which is a different and more defensible operation.
Questions people ask
Is voice emotion detection accurate?
The acoustic measurement is real: pitch, energy and speaking rate can be measured. The inference from those signals to a named emotion is contested and does not transfer reliably across individuals, cultures and contexts. There is no independently verified accuracy figure for voice emotion detection that holds across populations, and no standard benchmark to compare providers, so treat any vendor accuracy number as measured on its own dataset.
Is emotion detection legal?
It depends on where and how it is used. The EU AI Act prohibits emotion inference in workplace and educational settings under its prohibited practices provisions, applied from 2 February 2025, and that is an absolute prohibition rather than a conditions category. Other contexts are not banned, but using it to make decisions about individuals carries fairness and accuracy risk everywhere.
What is the difference between emotion detection and sentiment analysis?
Emotion detection infers a person's internal emotional state, usually from the sound of the voice, which is scientifically contested. Sentiment analysis classifies the language that was actually said as positive, negative or neutral, which is an observable artifact and a well-established task. Sentiment makes the smaller and more defensible claim, and the two are marketed interchangeably despite standing on very different ground.
What is affective computing?
Affective computing is the broader field concerned with systems that recognize, interpret or simulate human emotion. Emotion detection is the part of it that tries to read emotional state from signals such as the voice. The defensible version of the capability describes the acoustic measurement, uses it as a triage signal rather than a conclusion, and states its reliability limits plainly.
Last reviewed 31 July 2026. Written and reviewed by Dhruv Dholakia, founder of AsqVox.