Barge-in detection is capped by echo cancellation

What it is

A kitchen speaker is reading a recipe at volume; the user says “next page.” The microphone holds both leaked TTS from the loudspeaker and near-end speech. If echo is under-cancelled, the voice is thrown away as residual echo and barge-in never fires. If it is over-cancelled, the synthesizer’s own tail is treated as near-end speech and the system interrupts itself. AEC-limited barge-in detection is the split between “the dialogue manager allows barge-in” and “the microphone can separate the user from the speaker during playback.” The first can be always-on; the second is capped by residual AEC (acoustic echo cancellation) and double-talk detection.

Why it happens

In full duplex the playback signal is known; AEC subtracts that known path from the microphone. Room reflections, volume, and speaker–mic geometry move, and the path estimate lags. Residual echo inflates energy detection and produces false barge-in: the system stops with nobody talking. Raising the threshold to kill those false hits drops true near-end speech — especially short commands masked by the loudspeaker — and produces missed barge-in. During double-talk the adaptive filter has to freeze, or it will learn the user’s voice into the echo model and hear even less afterwards.

So “barge-in allowed” in dialogue management is not the same as audible during playback. Louder output, emptier rooms, farther-field devices leave less AEC margin, and detection becomes unusable. Half duplex (recognition off while playing) is a policy admission of that physical cap; the cost is that the user must wait out the prompt.

Studying it

In a controlled room, playback SPL × AEC on/off or quality for barge-in detection: play a fixed TTS passage, insert a short command on a marked word. Dependent measures: hits, false alarms (self-stops with no insert), detection latency, residual echo energy. Independent variables: loudspeaker level, mic distance, reverberation. Set dialogue management to “always allow barge-in,” so failures are acoustic.

In production logs, align user-onset time with actual stop-of-playback. If playback continues hundreds of milliseconds after onset and SPL is high, look at AEC before the intent model. A lab that plays TTS over headphones the microphone cannot hear will measure the AEC problem out of existence.

Where it stops holding

An earpiece takes playback out of the room; AEC load collapses and barge-in detection is no longer this bottleneck — endpointing and intent are. When the user has already turned playback down, near-end is relatively loud and weak AEC still passes. Extremely short overlap (a single “uh-huh”) may never clear a reliable energy gate even if the user heard themselves cutting in. Treating missed barge-in as “the user was not loud enough” pushes an acoustic failure onto the person and invites a raise-your-voice loop.

Applying it

• Split barge-in acceptance into two layers: whether dialogue allows it, and whether hit rate holds at the product’s target volume and room. Quiet near-field “allowed” is not enough.
• Test misses and self-stops separately on high-volume far-field skills (kitchen, car speaker). If self-stops are high, do not “fix” them by raising the energy threshold — true barge-in dies with them.
• Feed known TTS into AEC as reference; freeze adaptation during double-talk. On suspected near-end, stop playback first, then decide whether the utterance is valid input.
• How to check: in the target scene, set volume to the product’s high-ish default, barge in mid-prompt with “stop / next.” Log onset-to-stop and false alarms. Misses rising with volume and false alarms rising with reverb mean AEC is the cap, not the wording.

Related

• Same group: M3.09.2 After barge-in, tell correction from a new request · M3.09.3 Ignored barge-in makes people raise their voice
• Nearby: M3.04 Barge-in · C7.05 Noisy Environments · C7.12 Recognition Degradation in Noise
• Search terms: AEC-limited barge-in detection · acoustic echo cancellation · double-talk