Passing accessibility checks is not the same as being usable by disabled users

What it is

Accessibility checks—automated scans or manual audits—verify that the interface satisfies standard criteria, and criteria can only encode decidable minimums. An interface can pass every check and still leave screen-reader users lost in real tasks: labels present, order compliant, contrast sufficient, yet the information organization gives no way to form a plan of action. Conformance is proof of a floor, not proof of usability.

Why it happens

The gap comes from the expressive limits of standard criteria: they must be written to be decidable (alt text present or not, contrast passing or not), while usability depends on whole-system properties—whether the navigation logic holds together, whether the linearized flow is followable, whether error messages actually guide—that cannot become checkboxes. Automated scans cover even less: only the machine-decidable subset. So "checks passed" guarantees the absence of the crudest barriers, not that tasks can be completed; treating pass rates as a usability verdict is accessibility's most common pseudo-completion.

Studying it

Close the gap with task testing alongside assistive-technology users: recruit users with the target profiles (blind, motor-impaired, cognitively loaded), assign real tasks rather than "inspect this page," and observe completion, detours, and assistive-tech operating load. Diffing audit results against user-test results localizes the "compliant but hard" criteria—which passing items still obstruct actual use. Report the two layers separately: conformance status and task-usability status, never merged.

Where it stops holding

The claim does not demote checklists: they intercept a huge volume of basic barriers and are the only scalable layer; without them, user testing drowns in trivial findings. The reverse also holds—user testing without audits misses basic defects on rarely visited paths. The right combination is audits as the floor, user testing for depth, each with irreplaceable blind spots.

Applying it

• Fix a two-column format for accessibility acceptance: conformance results plus assistive-tech task results; a single passing column never licenses the word "accessible."
• Source user-test tasks from the same task library as ordinary usability testing (real goals), never "go find accessibility problems."
• Log criteria that passed audits but failed user tests into an internal high-risk-compliance list and re-check them first in later releases.

Related

• Same group: B5.13.1 Accessibility addresses a wider range of abilities; usability addresses achievement for a specified user group · B5.13.3 Improvements made for extreme ability conditions usually raise usability for ordinary users too · B5.13.4 Accessibility is usually legally constrained while usability is not, which changes how each is driven
• Nearby: J1 Accessibility and Inclusion · Q2 Usability Testing
• Search terms: accessibility audit · compliance gap · screen reader testing