Empirical Investigation of Accessibility Bug Reports in Mobile Platforms: A Chromium Case Study

Title of the Paper

Empirical Investigation of Accessibility Bug Reports in Mobile Platforms: A Chromium Case Study

Paper Information

• Topic Area: Accessibility research in mobile platforms, specifically focusing on accessibility bug reports in the Chromium project.
• Keywords: Accessibility bug reports, Google Chromium, mobile applications, open source, empirical study, Android, iOS, bug fix time, assistive technology

Research Background and Issues

• Identified Problems or Challenges:

  • Despite the availability of abundant resources and tools guiding developers to build accessible applications, most mobile and web applications still face accessibility barriers.
  • There is very limited research on how accessibility issues are handled and the time required for their resolution.
  • There is a lack of in-depth analysis on how developers respond to and fix Accessibility Bug Reports (ABRs).

• Significance:

  • Enhancing software accessibility is crucial for ensuring equal digital access for all users, including those with disabilities.
  • Understanding the prioritization and categories of accessibility issues can provide practical guidance for developers to improve their practices.

• Research Motivation and Related Work:

  • Previous studies have largely focused on developers' awareness and perceptions of accessibility issues, with few delving into the reporting and resolution process of these issues.
  • The authors use the Chromium mobile platform as a case study to explore the reporting frequency, fix time, and root causes of accessibility bug reports.

Solution

• Proposed Method or Solution:

  • Conduct an empirical study analyzing accessibility-related bug reports in Google Chromium.
  • Investigate three main research questions: ① Trends in the number of reports; ② Time required for fixes; ③ Root causes of the defects.

• Innovations:

  • The first systematic comparison of accessibility bug reports in open-source software across Android and iOS platforms, including an in-depth analysis of report distribution, prioritization, and fix times.
  • Development of a framework based on open coding to categorize the root causes of accessibility issues, providing a reference for developers.

• Implementation Steps and Key Techniques:

  1. Data Collection: Export bug reports related to the Chromium mobile platform (Android and iOS) from the Monorail bug tracking system.
  2. Data Cleaning: Remove test or irrelevant reports and tag accessibility-related reports with labels such as "A11y."
  3. Data Analysis:
     • Use statistical and visualization methods to analyze report trends, issue categorization, and fix times.
     • Apply open coding to semantically classify report content and summarize root causes.
  4. Priority Analysis: Evaluate how developers prioritize accessibility issues compared to non-accessibility issues.

Research Findings

• Specific Findings:

  • Report Distribution (RQ1):

    • The number of accessibility bug reports has increased significantly since 2013, with Android reports outnumbering those on iOS.
    • Report numbers peaked in 2014 and 2015, after which iOS reports declined while Android reports continued to grow.

  • Fix Time (RQ2):

    • Accessibility bug fixes take significantly longer than non-accessibility bug fixes (median fix time for Android: 52.5 days vs. 15.0 days for non-accessibility issues).
    • Accessibility issues are generally assigned lower priority compared to non-accessibility issues.
    • The fix efficiency on Android is better than on iOS.

  • Root Causes (RQ3):

    • Over 25% of reports involve suggestions for improving accessibility features.
    • Top-ranked defect causes include "design issues" (Android: 23.1%, iOS: 17.9%) and "assistive technology issues" (approximately 16%).
    • Specific problems focus on slider interactions, button labeling, and screen reader compatibility.

• Advantages Over Existing Solutions:

  • Systematically analyzes the types, fix times, and reasons behind developers' prioritization of accessibility issues.
  • Provides a comprehensive defect categorization method to help developers more efficiently identify and resolve similar issues.

• Experimental or Evaluation Results:

  • Accessibility-related issues account for around 4% of total reports, with Android contributing 3% and iOS 1%.
  • Wilcoxon test results indicate that the fix speed for Android reports is significantly faster than for iOS.

• Limitations and Future Directions:

  • The study only analyzes the Chromium project, which may not fully represent other open-source software projects.
  • Future work could expand to monitor other mobile platforms or consider multilingual scenarios.
  • Development of automated tools to detect accessibility bug reports based on keywords is a potential area for further research.