A Probabilistic Model and Metrics for Estimating Perceived Accessibility of Desktop Applications in Keystroke-Based Non-Visual Interactions

Document Title

A Probabilistic Model and Metrics for Estimating Perceived Accessibility of Desktop Applications in Keystroke-Based Non-Visual Interactions

Document Information

• Subject Area: Human-Computer Interaction (HCI), Assistive Technology, Accessibility Design
• Keywords: Perceived Accessibility, Usability, Blind Users, Screen Reader, Keyboard, Computational Model, Probabilistic Model, Desktop Applications

Research Background and Problem Statement

• Problem Description:

  • The accessibility experienced by visually impaired users in desktop applications is primarily determined by the logical layout and accessibility of UI elements. Currently, there is a lack of effective quantitative models to measure "perceived accessibility," which is particularly critical for non-visual users relying on screen readers and keyboard-based interactions.
  • Existing methods (e.g., accessibility consistency evaluations) fail to adequately reflect users' actual experiences and provide limited guidance for developers seeking optimization.
  • Researchers have identified the practical significance of systematically evaluating navigation complexity and design rationality among UI elements.

• Research Importance:

  • Desktop applications remain indispensable tools for visually impaired individuals in employment and educational environments. Enhancing perceived accessibility can significantly improve user experience and efficiency.
  • Traditional methods are overly generic and fail to help developers identify UI issues or make optimization decisions. Models that are easier to automate can shorten design and testing cycles.

• Research Motivation and Related Work:

  • This study builds on existing HCI theories (e.g., GOMS model) and probabilistic user behavior modeling methods, proposing finer-grained interaction efficiency evaluation metrics to address the shortcomings of current consistency tests.
  • While some literature attempts to quantify accessibility using cognitive models, these methods are cumbersome to apply and have high learning costs, limiting their practicality.

Solution

• Proposed Solution:

  • Develop a probabilistic model based on UI navigation structures to evaluate the complexity of keystroke paths between different UI elements.
  • Introduce three new metrics: Complexity, Coverage, and Reachability, to quantify the quality of user experience in keyboard interactions.
  • Create a tool that extracts UI hierarchy from application accessibility APIs and uses it as input to calculate the metrics. The tool is automated and requires minimal involvement from users or developers.

• Innovative Features:

  • Unlike traditional GOMS models, this method approaches the application holistically (rather than focusing on specific tasks) by modeling comprehensive UI navigation to predict user experience.
  • Introduces a weighted model based on user feedback data, allowing metrics to reflect personalized usage habits.
  • Provides an automated pre-assessment mechanism that does not require direct user participation, with the potential to identify and resolve accessibility issues early in the development cycle.

• Implementation Steps and Key Technologies:

  1. Develop a tool to extract UI tree structures using system UI automation APIs.
  2. Use the UI tree as a navigation graph to calculate the keystroke navigation cost between UI elements.
  3. Establish two transition models: uniform probability (default) and user-weighted probability (based on survey data).
  4. Generate Complexity, Coverage, and specific percentage Reachability metrics using these models.
  5. Validate the metrics across different applications and propose potential optimization strategies.

Research Outcomes

• Key Findings:

  1. Identified core factors influencing perceived accessibility for blind users, including navigation complexity, operational independence, and shortcut key consistency.
  2. Proposed three probabilistic model-based metrics (Complexity, Coverage, Reachability) to quantify the accessibility characteristics of different applications.
  3. Tested common desktop applications (e.g., Microsoft Word, Excel, Notepad) to quantify and compare their perceived accessibility and shortcut key learning effectiveness.

• Advantages Over Existing Solutions:

  • Compared to traditional manual evaluation methods, the metrics provide faster and more objective feedback for developers.
  • Can be integrated into continuous integration and automated testing, supporting developers in optimizing accessibility early in the design cycle.

• Experimental or Evaluation Results:

  • Eleven blind users participating in the tests found the metrics to accurately quantify their real-world experiences.
  • For example, MS Word and Notepad were compared, with Word offering richer functionality but requiring users to memorize more shortcuts for efficient navigation, as indicated by the Complexity metric.
  • Automated metric evaluations highlighted the significant reduction in navigation costs provided by common shortcuts (e.g., Word's "Ctrl+B"), particularly in applications with deeper navigation hierarchies.

• Limitations and Future Directions:

  • The model is focused on desktop applications and needs to be extended to mobile devices and touch-based interactions.
  • The representativeness of the metrics for user experience requires validation with larger user samples.
  • The range of tested applications is limited, and further verification of the metrics' universality is necessary.