Ability Heuristics for Conducting Accessibility Inspections

Paper Title

Ability Heuristics for Conducting Accessibility Inspections

Publication Info

• Topic area: Accessibility evaluation methods for interactive technologies.
• Keywords: Accessibility, heuristic evaluation, ability-based design, WCAG, usability heuristics, interactive technology, accessibility inspections, design process, accessibility quality.

Background and Problem

• Problem / challenge: Accessibility evaluations often rely on checklists that are low-level, numerous, and platform-specific, requiring specialized expertise and offering limited support for everyday designers and developers during the design process.
• Significance: Ensuring accessibility in interactive technologies is critical for inclusivity, especially as novel interaction modalities like virtual reality proliferate. Designers need accessible tools to evaluate and improve their designs without requiring extensive expertise.
• Motivation and related work: Prior methods like WCAG and usability heuristics are either too detailed and checklist-oriented or lack a focus on accessibility. Inspection methods, particularly heuristic evaluations, have proven effective for usability but have not been clearly established for accessibility. This paper builds on ability-based design principles to address this gap.

Solution

• Proposed approach: Ability heuristics—a set of nine heuristics designed to evaluate accessibility in interactive technologies, emphasizing the quality of accessibility features and flexibility across diverse disabilities.
• Novelty:
  1. Development of nine ability heuristics grounded in ability-based design principles and accessibility themes.
  2. Empirical evaluation of ability heuristics against WCAG and usability heuristics in a classroom study.
  3. Focus on accessibility quality and flexibility rather than binary compliance.
• Procedure and key techniques:
  • Two years of formative work analyzing student assignments on eliciting ability assumptions.
  • Brainstorming and iterative design sessions involving accessibility experts and individuals with disabilities.
  • Qualitative coding and refinement of heuristics based on themes from ability-based design, WCAG, and usability heuristics.
  • Evaluation through a classroom study with 37 design students using three inspection methods on three interactive technologies.

Results

• Concrete findings:
  • Students using ability heuristics identified 657 accessibility issues, emphasizing the quality of accessibility features and flexibility of interfaces.
  • Ability heuristics highlighted broader accessibility issues compared to WCAG and usability heuristics, such as incomplete accessibility features and inequitable experiences.
  • NASA TLX workload scores showed no significant differences in perceived effort across inspection methods, indicating feasibility of ability heuristics.
• Advantage over baselines:
  • Ability heuristics surfaced more nuanced issues related to accessibility quality and flexibility compared to WCAG and usability heuristics.
  • Students applied ability heuristics more evenly across the set compared to usability heuristics.
• Experiments / evaluation:
  • Between-subjects study with 37 design students evaluating Microsoft PowerPoint, Google Maps, and Amazon.com using ability heuristics, usability heuristics, and WCAG.
  • Data analyzed through thematic coding and statistical tests (e.g., Kruskal-Wallis, chi-squared).
• Limitations and future work:
  • Limited evaluation scope (three technologies, two device types).
  • No direct involvement of users with disabilities in the evaluation.
  • Future work should explore broader applications, integration with generative AI, and long-term impacts on product accessibility.

Summary

This paper introduces ability heuristics, a set of nine heuristics for conducting accessibility inspections of interactive technologies. Grounded in ability-based design principles, these heuristics emphasize accessibility quality and flexibility, moving beyond binary compliance measures. Empirical evaluation with design students demonstrated their effectiveness in identifying diverse accessibility issues and their feasibility for use by non-experts. While promising, further research is needed to validate their robustness across more technologies and contexts, and to explore their integration with emerging tools like generative AI.