Latte: Use-Case and Assistive-Service Driven Automated Accessibility Testing Framework for Android

Document Title

Latte: Use-Case and Assistive-Service Driven Automated Accessibility Testing Framework for Android

Document Information

• Subject Area: Accessibility testing and evaluation for Android applications
• Keywords: Android, accessibility, automated testing, assistive services, mobile applications, usability testing, inclusive design, Graphical User Interface (GUI) testing, TalkBack, SwitchAccess

Research Background and Issues

• Identified Problems and Challenges:

  • Over 15% of the global population has some form of disability, making mobile application accessibility crucial for them.
  • Existing automated accessibility evaluation tools are primarily based on predefined rules and fail to reflect the actual interaction experiences of users with disabilities.
  • Limitations of current technologies include: excessive warnings, lack of distinction between primary and secondary functionalities, and neglect of critical user interaction issues in assistive services.
  • Existing assistive tools (e.g., TalkBack or SwitchAccess) offer limited support for dynamic interfaces or custom controls.

• Research Importance:

  • Improving mobile application accessibility is a significant goal of technological innovation.
  • Effective accessibility testing can help developers enhance user experience and prevent accessibility barriers for a wide range of users with disabilities.

• Research Motivation and Related Work:

  • Previous methods, including static and dynamic analysis, fail to capture real user interaction issues effectively.
  • Dynamic analysis tools (e.g., Google Accessibility Scanner) require manual operation and cannot systematically cover all primary functionalities.
  • The motivation for this research is to develop a novel method that accurately reflects the actual interaction patterns of users with disabilities and identifies critical accessibility issues related to assistive services.

Solution

• Method and Solution:

  • A high-fidelity automated accessibility testing framework called Latte is proposed.
  • The key idea behind Latte is to leverage developers' existing functional tests, extract use cases, and automatically execute these use cases within assistive services (TalkBack and SwitchAccess) to detect accessibility issues.

• Innovations:

  • Integrates functional testing with accessibility testing using assistive services to detect accessibility issues from the perspective of user interactions.
  • Automatically executes application use cases while generating detailed reports (including error causes and reproduction steps).
  • Reduces the number of warnings developers need to address, providing more precise reports focused on defects in primary application functionalities.

• Implementation Steps and Key Techniques:

  • Step 1 - Test Analyzer: Analyzes the input GUI tests and translates them into user-perceivable use case specifications.
  • Step 2 - Use Case Executor: Executes use cases within the TalkBack and SwitchAccess environments by simulating user interactions via assistive service APIs.
  • Step 3 - Result Analyzer: Collects data during execution, including accessibility issues and interaction counts, and generates usability reports.
  • Utilizes dynamic program analysis, Android AccessibilityService extensions, and automated testing frameworks such as Appium.

Research Outcomes

• Specific Results:

  • Latte successfully detected numerous accessibility issues that Google Accessibility Scanner failed to identify.
  • Tested 20 real-world Android applications and discovered 32 issues that existing technologies could not detect.
  • Achieved 100% accuracy in use case execution (no false positives).

• Comparative Advantages Over Existing Solutions:

  • Latte focuses on critical user use cases rather than static screen scans, significantly reducing developers' workload on minor issues.
  • Compared to Google Accessibility Scanner, Latte identified more critical defects (e.g., dynamic layout issues, navigation path blockages, non-standard component problems).
  • Improved developer efficiency in fixing issues through more precise and actionable reports.

• Experimental and Evaluation Results:

  • On average, each use case execution required 11 direct interactions, while using assistive services required 43 and 51 interactions, respectively.
  • The Scanner generated 1,716 issue reports, of which only 18 were relevant; in contrast, Latte reported only the core accessibility issues for each use case.
  • Latte systematically categorized issues such as dynamic layouts, navigation loops, and non-standard implementations, providing solution recommendations.

• Limitations and Future Directions:

  • Latte does not cover accessibility issues for non-assistive service users (e.g., low-contrast text), requiring integration with other tools.
  • Future research will expand to include more assistive services and complex gestures, as well as conduct user studies to uncover undetected accessibility issues.