"It Brought Me Joy": Opportunities for Spatial Browsing in Desktop Screen Readers

Research Background and Problem Statement

• Identified Problems and Challenges:

  • The interaction experience of blind and low-vision (BLV) users with desktop websites is significantly inferior to that of non-BLV users (sighted users).
  • Current screen readers present web content linearly through audio narration, failing to convey the spatial layout of web pages. This operational method excludes the richness of visual semantics (e.g., layout), making it difficult for users to quickly locate or understand specific content positions.
  • Although some studies have explored tactile and audio feedback, these often require additional hardware or are incompatible with current assistive technology standards.

• Importance of the Problem:

  • Understanding webpage layouts is crucial for user navigation experiences, web design cognition, and broader digital inclusivity.
  • The current digital divide makes it difficult for BLV users to share the same frame of reference as sighted users, limiting collaboration and access to information.

• Research Motivation and Related Work:

  • BLV users have shown interest in visual semantics (Potluri et al., 2021) and have reported a stronger sense of spatial navigation on mobile platforms compared to desktops.
  • Although spatial interaction methods based on tactile or audio feedback (e.g., ScreenTrack and CollabAlly) exist, they fail to meet the needs of spontaneous browsing on dynamic websites and have not been extended to desktop screen readers.

Solution

• Proposed Method and Solution:

  • A Google Chrome extension tool, "WebNExt," is proposed to enable webpage layout perception through spatial input (directional key navigation) and spatial output (spatialized audio feedback based on stereo and pitch).
  • WebNExt builds on existing desktop screen readers (e.g., JAWS, NVDA, VoiceOver) to provide direction-based spatial navigation.

• Innovative Features:

  • Combines directional input and spatial audio output, allowing users to construct a "mental map" of webpage layouts through auditory perception.
  • Requires no additional hardware and is fully compatible with existing screen reader standards, enabling flexible and convenient operation.
  • Supports spatial interaction with dynamic websites, not limited to static or lab-constructed web pages.

• Implementation Steps and Key Technologies:

  1. Technical Probe Design: WebNExt maps the midpoint positions of webpage elements to audio stereo (left-right position) and pitch changes (up-down position).
  2. Input Navigation Mechanism: Uses directional keys or WASD keys to navigate and locate the nearest page element in the chosen direction.
  3. Audio Spatialization: Utilizes the Web Audio API and meSpeak.js library to render spatial audio of page elements in real time.
  4. Scrolling Feedback: Plays a series of tones when users scroll the webpage to indicate the direction and extent of scrolling.

Research Outcomes

• Specific Results:

  • In both laboratory studies and a five-day field study, participants generally reported that WebNExt provided an intuitive and enjoyable spatial browsing experience.
  • BLV users gained a closer understanding of webpage design and expressed preferences for simplicity and intuitive layouts.
  • Users viewed WebNExt as a tool to reduce the digital divide, offering the potential to establish a shared frame of reference with sighted individuals (e.g., family, colleagues).

• Comparison with Existing Solutions and Advantages:

  • Compared to existing tactile or audio feedback-based methods, WebNExt's advantages include no need for additional hardware and strong integration with existing screen readers.
  • Provides broad support for dynamic websites, rather than being limited to experimental static sites.

• Experimental and Evaluation Results:

  1. In laboratory studies, all users were able to accurately perceive page layouts through WebNExt, a capability not achievable with traditional screen readers.
  2. Field studies showed higher spatial navigation value ratings for task-oriented websites (e.g., Google Search and Wordle).
  3. Several participants noted feeling "immersed" and "enjoying the experience," particularly in scenarios like online gaming.

• Limitations and Future Directions:

  • Limitations:
    • WebNExt currently supports only the Chrome browser; cross-browser implementation is needed for real-world deployment.
    • It does not currently support deaf-blind users.
    • Users must disable their personal screen readers to activate WebNExt, which could be optimized.
  • Future Directions:
    • Collaborate with mainstream screen reader developers to natively integrate spatial browsing functionality.
    • Develop a tactile version to support deaf-blind users.
    • Expand the scale of experiments to include longer durations and more diverse user groups.