What Do We Mean by "Accessibility Research"? A Systematic Review of Accessibility Papers in CHI and ASSETS from 1994 to 2019

Title of the Paper

What Do We Mean by "Accessibility Research"? A Literature Survey of Accessibility Papers in CHI and ASSETS from 1994 to 2019

Paper Information

• Domain: Assistive Technology and Accessibility Research
• Keywords: Accessibility, Assistive Technology, Disability, Literature Survey, Design Methods, Research Trends

Research Background and Issues

• Identified Problems or Challenges:

  • The field of accessibility research has grown significantly in recent years, but historically lacks comprehensive literature reviews.
  • Existing studies often focus on specific communities or topics, lacking evaluations of broader trends.
  • Certain prevailing practices and focal points in the field need reassessment, such as issues of representation among disabled populations and the generality of technological interactions.

• Significance:

  • Summarizing and critically reflecting on the past and current directions of accessibility research can help identify research gaps and enhance diversity in the field.
  • Provides clearer direction and guidance for new researchers entering the domain.

• Motivation and Related Work:

  • Existing studies have conducted focused reviews on specific subfields of accessibility (e.g., autism, visual impairments), but lack a comprehensive review of the entire domain.
  • The authors aim to fill this gap by analyzing accessibility research presented at the major conferences CHI and ASSETS.

Solution

• Proposed Method/Approach:

  • The authors created a dataset consisting of 836 papers from two conferences over 26 years (1994–2019), with qualitative analysis of 506 papers from the last decade (2010–2019) and quantitative analysis of overall trends.
  • The included studies were coded across multiple dimensions, such as target user groups, research problems, and methodologies, and analyzed for keyword trends over time.

• Innovations:

  • Combined qualitative and quantitative analysis methods to systematically investigate research papers from the past 26 years.
  • Offered new insights into the focus, methods, and practices of accessibility research.
  • Openly shared reusable datasets and code to support further meta-analyses in the field.

• Implementation Steps and Techniques:

  1. Dataset Creation:
     • Papers from CHI and ASSETS (1994–2019) were retrieved using the ACM Digital Library.
     • Data was cleaned, and relevant papers were manually and programmatically filtered.
  2. Coding and Analysis Methods:
     • Three researchers developed a coding system with 10 categories (e.g., community focus, research goals, method types).
     • Each paper was coded by at least two authors to ensure consistency, with discussions to resolve discrepancies.
  3. Data Visualization and Interpretation:
     • Trends in community focus, keyword frequency distributions, and other metrics were analyzed.

Research Findings

• Specific Results:

  1. Identified concentration and gaps in research:
     • 43.5% of papers focused on visual impairments, while topics like mental health and chronic illnesses were underrepresented.
  2. Common characteristics of studies:
     • Most studies employed interviews, usability testing, and controlled experiments as primary methods.
     • The median number of participants in experiments was 13, reflecting challenges in recruiting users in accessibility research.
  3. Keyword and theme changes over time:
     • For example, "Web" was the most frequently mentioned technical keyword from 1994–2019, while recent years have seen a rise in terms like "mobile devices" and "3D printing."

• Comparison with Existing Solutions:

  • This study stands out for its comprehensive focus on the entire field, rather than niche subfields, using a combined qualitative and quantitative perspective.
  • It specifically addresses sensitive issues such as participation methods and proxy roles in disability research.

• Experimental or Evaluation Results:

  • Accessibility research has grown significantly, with related papers accounting for 7.8% (approximately 8%) of total CHI papers in recent years.
  • Although representation of participant groups is gradually expanding, certain communities remain significantly underrepresented.

• Limitations and Future Directions:

  • The study is primarily based on CHI and ASSETS conference data, excluding broader journals and conferences.
  • Limitations in authorship and coding methods may affect the flexibility of certain classifications.
  • Future research should explore areas such as mental health, multiple disabilities, and effective integration of disabled researchers' perspectives.
  • The authors suggest further development of data-driven research contributions, including applications of AI fairness in accessibility.