Examining Visual Semantic Understanding in Blind and Low-Vision Technology Users

Document Title

Examining Visual Semantic Understanding in Blind and Low-Vision Technology Users

Document Information

• Subject Area: Computer Science and Engineering, Accessible Technology Design
• Keywords: Accessible Technology, Visual Semantics, Screen Readers, Blind and Low-Vision Technology Users, User Interface Navigation, Creative Design, User Research

Research Background and Issues

• Issues and Challenges: How blind and low-vision (BLV) technology users access and understand the visual semantics of user interfaces (UI) remains underexplored. Current screen readers primarily linearize UI information, limiting BLV users' ability to comprehend and interact effectively with spatial characteristics of the UI.
• Significance: Visual semantic information can provide BLV technology users with more efficient interface interaction capabilities while enhancing collaboration with sighted users. However, there is currently a lack of convenient methods for BLV users to access and utilize this information.
• Research Motivation: The authors aim to explore BLV users' needs, understanding, and the impact of visual semantic information on UI design. The goal is to propose directions for screen reader development and provide design recommendations.

Solution

• Methodology: The authors conducted a three-part qualitative study, including interviews, task analysis, and low-fidelity prototype-based UI reconstruction experiments, engaging with 10 BLV participants to investigate their understanding of visual semantics.
• Innovations:
  1. Pioneering an in-depth exploration of BLV users' understanding of visual semantics in everyday technology use scenarios.
  2. Introducing low-fidelity prototype design into the field of blind technology to collect user modeling and expression methods for visual semantics.
• Experimental Steps and Techniques:
  1. Semi-structured Interviews: Exploring BLV users' understanding, usage, and needs regarding visual semantics.
  2. Screen Reader Task Analysis: Observing interactions involving visual semantics during routine tasks on smartphones and desktop devices.
  3. Low-Fidelity UI Prototype Design: Using clay and tactile materials to reconstruct user-selected interface layouts, documenting thought processes and operational segments.

Research Findings

• Specific Findings:
  1. BLV technology users have a certain degree of understanding of visual semantic concepts and learn these concepts through trial and error or seeking assistance during daily tasks.
  2. Visual semantic information is used and valued significantly more in smartphone application scenarios than in desktop environments.
  3. Screen readers provide insufficient support for exploring visual semantics, with missing information and inconsistent feedback in UIs being major obstacles.
• Advantages: Compared to existing screen reader functionalities, the study proposes improved design suggestions, such as introducing a "visual semantic detail mode" to enhance user information acquisition and optimizing navigation features based on spatial information.
• Experimental or Evaluation Results:
  • Analysis of participants' cognitive accuracy regarding size, shape, and layout structure during UI reconstruction tasks revealed varying levels of accuracy across different interface types.
  • Some participants demonstrated a certain degree of spatial layout understanding even without visual support during UI tasks.
• Limitations and Future Directions:
  • Limitations: Small sample size and participants' high technical proficiency may not fully represent novice user groups; experimental design requires further validation for generalizability.
  • Future Directions:
    1. Develop screen reader functionalities for describing UI visual semantics.
    2. Provide prototype tools tailored for BLV users to enhance operability in the design process.
    3. Explore AI-based semantic description generation technologies to optimize visual support.

Conclusion and Discussion

The document provides clear research directions for screen reader development and emphasizes the importance of improving BLV users' interface interaction and visual design capabilities. This study not only introduces new perspectives to accessible technology design but also highlights the potential and necessity of supporting complex visual tasks for technology users.