The system must decide when the user has finished speaking
Aliases: endpointing · VAD · end-of-utterance · speech segmentation
What it is
Voice input has no key-up that means “submit.” The system has to decide when the user has finished before it can close a stretch of audio into one recognition request. That decision is endpoint detection; the front-end often uses voice activity detection (VAD) to mark speech onset and a suspected offset. Once end-of-speech is declared, capture usually stops and the hypothesis is committed. The question is whether this breath of speech has ended, not whether the sentence is semantically complete.
Why it happens
Continuous audio has no native period. A recognizer consumes features frame by frame; if it never closes a turn, latency piles up and partial hypotheses never become an executable final one. VAD first labels frames as speech or non-speech; after speech, a long enough run of non-speech is taken as the endpoint. Onset has to be judged as well: too early and room noise enters the buffer; too late and the first syllable is cut. Endpointing is the device that turns analog airflow into discrete turns. It is not the wake-word guard: the guard decides whether to enter listening; the endpoint decides at which frame this listening turn ends. Push-to-talk hands the endpoint to a release; hands-free use has no such gesture, so an acoustic judgment is required.
Studying it
Evaluate detectors on corpora with hand-marked start and end times: onset error, offset error, speech-frame hits and false alarms. Tasks should include complete sentences, long sentences with subordinates, and an immediate stop after the last word. Dependent measures include phonemes lost at the edges and extra wait from true end to system commit. Read scripts yield unrealistically clean boundaries; filled pauses and breaths in spontaneous talk change how the same detector behaves.
Where it stops holding
Hold-to-talk, release-to-send interfaces do not depend on automatic endpointing; the endpoint is contact loss. When a dictation box is already text, and the user commits with a keyboard return, endpointing is not the decision maker on that path. Meeting transcription often slices a continuous stream and does not treat every silence as “the user finished a command to the system.” Very short acknowledgements (“ok,” “mm”) and coughs or inhales are easily confused with speech frames, and the detector’s boundaries there are unreliable.
Applying it
- For hands-free commands and dictation, design three explicit states—capturing, listening, committed—and make commit correspond to one endpoint decision.
- Give people an end action that does not rely on acoustics: a button, a gesture, or keyboard confirm, so they are not stuck waiting when the detector hangs.
- In acceptance tests, look at clipped first syllables and over-long waits at the end, not only at whether the transcript is correct.