Attribution when several people speak at once
Aliases: cocktail party · overlapping speech · speaker attribution · source separation
What it is
When more than one person vocalises in a room, the microphone receives a sum. The system has to decide which stretch was addressed to the device and which is bystander talk—an attribution problem. Even if the SNR number looks fine, executing a neighbour’s sentence as a command is a failure. The cocktail-party effect is how humans select one stream in overlap; machines have no default tag for “this was meant for me.”
Why it happens
A single-channel mixture stacks two talkers’ phones on the same frame; ordinary ASR decodes a mash, or favours whichever stream has more energy. A far-field array can roughly separate sources by direction of arrival and still not know which stream is the owner. Wake words, speaker verification, and visual “facing the device” cues are used as gates: only a stream that passes them enters command parsing. When overlap hits mid-command, the tail may be contaminated by another person’s syllables and look like a recognition error whose root is a swapped source. Failed attribution also ships bystander content into cloud recognition; it is not only a functional miss.
Studying it
Use two-talker overlap: mix a target command with bystander chat at several overlap ratios, and measure whether the command is still attributed correctly and whether bystander content enters the transcript. Independent variables include angular separation, level difference, whether a wake word came first, and whether a voiceprint is enrolled. Report “bystander speech was executed” separately from “the target command was polluted.” Tests that add stationary noise to one talker measure the attribution problem away.
Where it stops holding
Call-time denoisers face one talker plus noise, not two speech sources. Meeting transcription aims to keep every speaker and run diarization—the opposite of a device command that should hear only the owner. Child voices and television speech are unstable as voiceprints. A counter microphone at a face-to-face kiosk may intentionally accept anyone; that attribution policy is the reverse of a living-room speaker.
Applying it
- For command products, default to not executing under overlap unless a wake or voiceprint marks the source as the current user.
- On far-field devices, combine array direction with a visible “listening this way” cue so talk on the opposite sofa is not taken as a command.
- Put “bystander speaks first” and “mid-command interruption” in acceptance tests and count wrongful executions, not only single-talker WER.