Understanding the Feasibility of Auditory Hand-Steering Guidance for Blind and Low-Vision People

Paper Title

Understanding the Feasibility of Auditory Hand-Steering Guidance for Blind and Low-Vision People

Publication Info

• Topic area: Assistive technology for blind and low-vision individuals focusing on auditory hand-steering guidance.
• Keywords: Blind and low-vision, hand steering, auditory guidance, VERBAL, Follow-Your-Finger, assistive technology, sonification, spatial navigation, human-computer interaction, accessibility.

Background and Problem

• Problem / challenge: Blind and low-vision (BLV) individuals face challenges in navigating their hands to targets while avoiding hazards due to the lack of visual feedback. Current assistive technologies either focus on reaching guidance or rely on impractical haptic devices, leaving a gap in practical hand-steering solutions.
• Significance: Effective hand-steering guidance could enable BLV individuals to perform everyday tasks independently and safely, such as handling kitchen tools or retrieving items in cluttered environments.
• Motivation and related work: Prior work explored haptic and auditory guidance for hand navigation, focusing on reaching tasks rather than steering. Haptic methods using robots are impractical for daily use, and auditory methods have yet to address steering guidance. This paper aims to fill this gap by developing audio-only hand-steering solutions.

Solution

• Proposed approach: Two auditory hand-steering guidance methods—VERBAL (spoken directional cues) and Follow-Your-Finger (FYF, sonification-based cues)—implemented on a Meta Quest 3 headset.
• Novelty:
  1. Development of the first auditory hand-steering guidance methods for BLV individuals.
  2. Quantitative evidence showing FYF’s feasibility and VERBAL’s limitations for steering tasks.
  3. Qualitative insights into improving FYF usability and identifying scenarios where VERBAL’s less precise but familiar approach may be effective.
• Procedure and key techniques:
  • VERBAL provides spoken instructions (e.g., "forward," "left") based on the angle between the fingertip and the next position along the path.
  • FYF modulates the volume of an 880 Hz tone to indicate alignment between the fingertip direction and the intended path direction.
  • Both methods were evaluated through a two-phase user study with 12 BLV participants, involving controlled tasks and realistic item acquisition scenarios.

Results

• Concrete findings:
  • FYF achieved a 4.17% error rate in WIDE Width conditions, with six participants preferring FYF achieving error rates of 1.39% (WIDE) and 2.78% (MODERATE).
  • VERBAL had higher error rates (24.6% in WIDE Width) and faster completion times (7.39s on average) but lacked precision.
• Advantage over baselines: FYF demonstrated lower error rates compared to VERBAL, particularly in conditions requiring higher steering accuracy.
• Experiments / evaluation:
  • Phase 1: Controlled study with predefined tunnels varying in width and amplitude.
  • Phase 2: Realistic item acquisition tasks in Desk and Shelf environments, with think-aloud protocols and exit interviews.
  • Metrics: Error rate, deviation, and completion time.
• Limitations and future work:
  • FYF’s sonification cues were difficult for some participants to interpret during the finding phase, leading to higher error rates in MODERATE Width conditions.
  • VERBAL’s zig-zag movement patterns and cognitive load limit its suitability for precise steering.
  • Future work should enhance FYF’s audio cues, explore VERBAL’s effectiveness in low-accuracy scenarios, and test ecological validity in real-world tasks.

Summary

This paper introduces two auditory hand-steering guidance methods—VERBAL and FYF—to support BLV individuals in navigating their hands through 3D space. FYF demonstrated lower error rates and greater precision, while VERBAL provided faster but less accurate guidance. A user study with 12 BLV participants highlighted FYF’s promise for safety-critical tasks and VERBAL’s potential for less demanding scenarios. The findings establish a foundation for developing practical, audio-based steering guidance systems using off-the-shelf devices, with future work needed to refine the designs and validate their applicability in real-world environments.