Variation within a disability type is large; a few participants do not cover the range

What it is

“We tested with blind users” is not a sample conclusion. Some people have no vision, some have residual vision; some learned JAWS in school, some turned on VoiceOver last week; some live on braille, some run speech at 80%; some combine magnification and a reader. Within-disability variation is larger than the homogeneity a type label implies. Two or three participants under one label cover their strategies, not the type’s full set of uses.

This is not “so skip user testing.” It is: after you have tested users, write the extrapolation in terms of strategy, not diagnosis.

Why it happens

Assistive technology translates one sensory limit into many paths. Once paths fork, interface failure points stop being shared: heading-jump is a primary weapon for fluent web users and almost unused by someone who mainly explores by touch on a phone; a high-contrast palette helps low vision and can hurt some photosensitive or colour-vision conditions. Treating “blind” as a persona takes a pass on the first path as a pass for the type.

The second layer is intersection. Motor plus cognitive, Deaf plus low vision, are more common than textbook single types. Recruiting on one label leaves intersection failures out of the sample. Age, digital fluency, and first-time use of this product often predict abandon better than the disability label. A handful of participants are almost always skewed on those axes — people willing to come to a lab are already a non-random use.

Studying it

Stop recruitment at the diagnosis name and log strategy: primary AT, braille or not, speech rate, magnification, voice control, digital fluency, age at onset. Analyse failures by strategy cluster, not by averaging “visual impairment N=4” into one line. If only one strategy could be recruited (all fluent NVDA users), write uncovered strategies as an explicit gap, rather than inflating N to fake saturation.

Independent variables: strategy cluster (not ICD label), whether the sample crosses clusters. Dependent variables: whether failure points change by cluster, error rate of extrapolating from one cluster to another.

Where it stops holding

Some issues are stable across strategies (a keyboard trap, a nameless button); a few participants reproduce them reliably, and within-type diversity is not the goal for those bugs. Early exploratory tests with two people on one strategy to find large pits are reasonable; treating that as conformance evidence is not. Rare strategies (Deafblind with sign as a first language, eye-gaze plus switch) are expensive to recruit; use targeted follow-ups rather than assembling a persona spectrum every time. Writing “everyone is different so testing is impossible” uses heterogeneity as a waiver.

Applying it

• Write recruitment in strategies (“uses a screen reader daily on web forms”), not only “visually impaired.”
• List the strategy clusters the plan intends to cover, and after the study mark which are empty; empty clusters must not be written as “visual impairment covered.”
• Phrase findings as “among fluent desktop reader users…,” never as “blind users could all complete this.”
• How to check: open the last user-test demographics table. If it has disability type and no AT or strategy, the sample was swallowed by the label. Check whether finding sentences are strategy-bounded; unbounded with N≤3 means a handful of people were treated as the set.

Related

• Same group: J5.14.1 Automated tools and expert review cannot replace testing with AT users · J5.14.3 Test on the user's own device and AT, not a lab-standard setup · J5.14.4 Paid disabled participants belong in early design, not only acceptance
• Nearby: J1.08 Permanent, temporary and situational disability · J5.07 Compatibility testing
• Search terms: within-disability variation · heterogeneity of disability · AT strategy