C7.10.1Streaming partial ASR resultsdesignresearch

Partial results show not-yet-final text while the user is still speaking

Aliases: interim results · streaming ASR · partial hypothesis

What it is

Streaming recognition emits words before speech has ended. Those words are partial or interim results: the current best hypothesis on the prefix, not yet finalized. Characters appearing on screen come from this unsealed hypothesis, not from committed text.

Why it happens

An encoder consumes audio in chunks; the decoder keeps a prefix search on each chunk (often an RNN-T or similar streaming model). Each chunk updates the prefix hypothesis, which the client can render. Time resolution depends on chunk size and the model’s latency constraint: smaller chunks emit earlier and jitter more. Partials exist because interaction cannot wait for the utterance endpoint before the first glimpse. Unlike an n-best list, the screen usually draws one current-best path. Unlike a final result, words on that path can still be rewritten by later audio. Partials are a live view of recognizer state, not of document state.

Studying it

Log time from speech onset to each word’s first appearance, and whether that word survives in the final. Independent variables include chunk size, network round-trip, and on-device versus remote decode. Dependent measures include time-to-first-character and how often partials jitter. Have people dictate with partials visible versus finals only, and count interruptions to correct mid-speech. Offline whole-utterance WER is not a substitute for prefix behaviour.

Where it stops holding

Offline batch recognition has no partials. Push-to-talk that decodes only on release can choose not to show intermediate words. Very short commands may end before a partial is drawn, so this layer is almost invisible. When the network dies, partials freeze on a prefix and look final even though the stream is dead.

Applying it

  • While speaking, draw the current prefix in the field and make clear it is a hypothesis in progress.
  • Measure and budget time from onset to first character, not only final WER.
  • On a dead stream, do not treat the last partial as committed text until an endpoint or the user confirms.

Related

  • Same group: C7.10.2 Partials may be revised or wholly replaced and need not match the final · C7.10.3 Showing partials live shortens perceived latency even if the final still waits · C7.10.4 Partials must not look like editable final text; style them as a distinct state
  • Adjacent: C7.02 Endpoint Detection · C7.16 Visible Feedback for Voice Input
  • Search: partial results · streaming ASR · interim hypothesis

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/C7.10.1