A Large-Scale Longitudinal Analysis of Missing Label Accessibility Failures in Android Apps

Title of the Paper

A Large-Scale Longitudinal Analysis of Missing Label Accessibility Failures in Android Apps

Paper Information

• Subject Area: Accessibility evaluation in human-computer interaction and mobile applications
• Keywords: Mobile app accessibility, longitudinal analysis, missing labels, Android apps, accessible design, data mining, accessibility standards, quantitative research, interface evaluation, user studies

Research Background and Issues

• Problem or Challenge: Many Android apps fail to properly implement accessibility standards, particularly lacking support for labeling image elements, which prevents users relying on screen readers from accessing certain app functionalities.

• Significance: Mobile applications are ubiquitous in modern life, but inadequate accessibility support in apps can create barriers for users with disabilities, exacerbating social inequality.

• Research Motivation: Previous accessibility studies have mostly relied on single snapshots, lacking systematic research from a long-term perspective. Longitudinal studies in web accessibility have provided valuable insights, prompting this study to adopt a similar approach for mobile applications.

Solution

• Method or Solution: The authors developed a crawler system to automatically collect and analyze data from 312 popular Android apps over a 16-month period, identifying, quantifying, and tracking accessibility issues related to missing labels.

• Innovations:

  1. Proposed a novel approach to track multiple versions of the same app and the long-term changes in its interface elements.
  2. Introduced and defined the analytical standards of "screen equivalence" and "element equivalence," enabling large-scale longitudinal comparisons of accessibility data.

• Implementation Steps:

  1. Data Collection: Monthly snapshots of app interfaces were obtained from the Google Play Store between December 2019 and March 2021 using a crawler system.
  2. Accessibility Detection: Google's Accessibility Testing Framework was used to detect missing labels in image elements (e.g., ImageView and ImageButton).
  3. Data Analysis: Statistical regression analysis was conducted across multiple dimensions (e.g., app download volume, app category, developer organization), examining trends in systemic improvements and accessibility regressions.

• Key Technologies:

  • Android crawler development
  • Machine learning-assisted automated data detection and preparation
  • Mixed Logistic Regression models for handling data variance

Research Outcomes

• Specific Findings:

  1. Quantitative Findings:

     • 55.6% of image elements had missing label issues.
     • Accessibility of ImageView elements showed no significant improvement, while ImageButton elements saw some progress.
     • 8.8% of screens required navigation through at least one unlabeled element to be accessible.
     • No significant positive correlation was found between app accessibility issues and download volume, with most apps failing to improve accessibility despite increased popularity.
     • Large software organizations (e.g., Google, Microsoft) demonstrated more effective accessibility improvements compared to other entities, though issues persisted.

  2. Qualitative Observations:

     • Systemic Accessibility Improvements: For example, Zillow's two apps achieved overall interface improvements by fixing issues in shared code.
     • Incomplete Accessibility Improvements: Some developers failed to uniformly add descriptive labels to all interface elements.
     • Interface Redesigns: Redesigns posed risks of introducing new accessibility issues.
     • Accessibility Regression: Certain elements that initially had labels lost support in subsequent updates.

• Advantages:

  • Systematic: The first study to track and quantify changes in app accessibility longitudinally.
  • Practical: The findings provide directions for developers to optimize tools and improve processes.

• Limitations and Future Directions:

  1. The study focuses solely on the Android platform; methods for data collection and analysis compatible with iOS and other platforms need to be developed.
  2. The dataset does not cover all types of interface elements (e.g., dynamically loaded content).
  3. Future studies could frequently track key update versions to better understand developer behavior trends.

Conclusion

This paper provides an in-depth analysis of accessibility challenges in Android app development, revealing widespread issues while offering critical insights for improving developer tools and best practices. The combined quantitative and qualitative longitudinal research approach can be extended to broader accessibility research domains.