M4.09.3training-data gap not fixable by promptsdesignresearch

A training-data gap cannot be patched with prompts

Aliases: speak-clearly fallacy · corpus gap · prompt cannot fix ASR

What it is

Onboarding copy that says “please use Mandarin, short sentences, enunciate,” or a repair prompt that says “say it again, more slowly,” cannot supply the child and older-adult speech missing from the training distribution. A data gap is a coverage problem: acoustic statistics never saw that fundamental-frequency range or that pause structure; language statistics never saw that syntax. A prompt asks the speaker to become the missing data. Gaps shrink by collection and adaptation. They do not shrink by writing a better sentence that assigns the blame. This is not “metrics must be reported by cohort” — that is how to evaluate. This is what follows once evaluation has already seen the gap: wording will not save the model.

Why it happens

A recognizer is a conditional distribution: high probability on acoustic–language pairings it saw in training, low on ones it did not. Child and older speakers have long been thin in public and industrial corpora, not because they speak “nonstandard,” because sampling did not cover them. Prompts can change user-side strategy: shorten, step closer, swap words. Those strategies sometimes push an utterance toward the training manifold (a short command looks more like adult command data than a story does), and that coincidence gets filed as “the prompt works.” For utterances that remain off-manifold — preschool F0, older retrieval timing, developing morphology — slower speech and different words do not change the vocal tract, and do not invent phone realizations the model has never seen. “Please speak clearly” after a miss sends an out-of-distribution point through the same model again. Cohort splits in fairness evaluation exist to expose this residual: if the gap does not close under a changed prompt condition, the gap is not in the wording.

Studying it

Same child or older speakers, two prompt conditions: speak naturally versus instructed to be clear, short, close. Ask whether stratified error shrinks to indifference against an adult control. If a residual remains, the gap is in the model. Add a third arm: the same prompts on a model adapted with that age band’s data, and see which arm closes the gap — that is what separates wording from data.

Do not use only cooperative lab participants. People willing to slow down and re-record are already performing the adaptation the prompt asks for; writing their success as product strategy excludes people who will not or cannot “talk like the training set.” Error analysis should split “after the prompt, the sentence is now in-vocabulary for the training set” from “after the prompt, the acoustics are still outside it.”

Where it stops holding

If the task is already a short command and the user is already inside the vocabulary, “use these words” is teaching a grammar, not filling an acoustic gap. Severe dysarthria needs a dedicated model and another input channel; collecting a bit more “older speech” will not cover it. A prompt can change channel (“type this on the phone”) — that is abandoning voice, not repairing recognition. Billing every miss to the gap will miss wake-word hardness, room noise, and early endpointing, which do not wait on more child hours.

Applying it

  • Do not write “please speak clearly” on failure. Write the next channel (look at the screen, tap, let an adult speak) or enter an age-specific recognition path.
  • If onboarding can only teach “use short sentences,” first check whether the skill already has training coverage on the target age band. No coverage: do not mark the skill child-ready.
  • Collection plans list minimum hours and task kinds (spontaneous, command, proper names) per age band. Write the gap as a data debt, not as a copy todo.
  • How to check: the same child or older users, natural versus “clear short sentence.” If stratified error still splits clearly, stop polishing the prompt and add data or change the model.

Related

  • Same group: M4.09.1 Children's acoustics and syntax both leave adult norms · M4.09.2 Older adults' rate and pauses trigger early turn decisions
  • Nearby: M4.04 Recognition Differences Across Populations · C7.04 Accent, Dialect, and Code-Switching · M2.04 Error-Recovery Wording
  • Search terms: ASR fairness · training-data gap · cohort evaluation

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/M4.09.3