Training-data distribution determines whose speech is recognized well

What it is

Whose speech an engine recognizes well depends first on whom the training data heard. Acoustic models, language models, and lexicons are estimated from corpora; if a speaker’s accent, age, gender, channel, or register is sparse there, word error rate at test time is systematically higher. Population gaps are not a fairness overlay added at decode time. They are a distribution fact: the model fits the voices it was sampled on, not “everyone’s voice.”

Why it happens

Supervised or self-supervised pretraining minimises prediction error on the seen distribution. Most public and industrial corpora lean toward standard varieties, adults, near-field, read or broadcast speech. Child vocal tracts, older voices, non-majority accents, telephone bandwidth, and noisy far-field have less weight in the gradient, so the decision boundary tightens around majority speakers. Language models likewise favour written or standard-colloquial order and assign low probability to syntactic patterns common in another group. The same “universal” model therefore yields different WER on different demographic slices with identical hardware. That is a different layer from one person misspeaking, or one sentence happening to be a homophone.

Studying it

Report WER and intent error stratified by speaker attributes, not only a pooled mean. Attributes should include accent or dialect labels, age band, gender, device, and signal-to-noise ratio. Crowdsourced corpora can be sliced but are noisy in labels; compare them with controlled read speech. When auditing a commercial API, hold script and recording conditions fixed across groups so content difficulty is not confounded with speaker attributes. Recruiting only campus-accented participants measures “our model is accurate” and misses the distribution problem.

Where it stops holding

Domain mismatch (medical terms, place names) also raises error rates; that is a lexicon problem, not necessarily speaker coverage. For one speaker, a microphone or room change can shift the channel more than accent does. Light speaker adaptation can pull an individual back toward the boundary, but it does not close the out-of-the-box gap for users who never enroll. Blaming every gap on “not enough data” ignores that the decode lexicon and language model can also reject how a group talks.

Applying it

• Set ship gates on slices, not on a weighted overall WER; the worst-covered slice decides whether this can be a general-purpose input.
• Write coverage of accent, age, and channel into the collection plan; fill missing slices before piling on more standard speech.
• Until slice error is acceptable, give those speakers a way to finish the task that does not depend on this recognizer.

Related

• Same group: C7.04.2 Recognition degrades on mixed Chinese–English and dialect words · C7.04.3 Accuracy gaps exclude particular populations · C7.04.4 Mid-utterance language switches require live language identification · C7.04.5 A wrong language decision forces the rest of the sentence through the wrong phonology · C7.04.6 Dialects share vocabulary with the standard but differ in pronunciation, so they are misread as near-homophones · C7.04.7 Accent and dialect gains depend on training data from those speakers, not on algorithms alone
• Adjacent: C7.03 Types of Recognition Errors · C7.12 Recognition Degradation in Noise
• Search: training data distribution · demographic WER · speaker coverage