From Selfie Stick to Virtual Cane: Enabling Blind Exploration through Mobile Virtual Reality

Paper Title

From Selfie Stick to Virtual Cane: Enabling Blind Exploration through Mobile Virtual Reality

Publication Info

• Topic area: Accessibility in virtual reality for blind or low-vision (BLV) individuals.
• Keywords: Mobile VR, accessibility, blind navigation, virtual cane, spatial audio, haptic feedback, cognitive mapping, orientation and mobility, pseudo-haptics, virtual environments.

Background and Problem

• Problem / challenge: Traditional VR systems rely heavily on visual input and are largely inaccessible to BLV individuals. Existing accessible VR systems often require specialized hardware, lack portability, and fail to support embodied cane-based exploration.
• Significance: Making VR accessible to BLV users can enable safe spatial exploration, support orientation and mobility (O&M) training, and foster participation in mainstream VR applications.
• Motivation and related work: Prior work has explored nonvisual locomotion, haptic and auditory feedback, and VR for O&M training, but these systems often depend on head-mounted displays or custom devices, limiting accessibility. Mobile devices, while widely adopted by BLV users, have not been fully leveraged for embodied cane-based VR exploration.

Solution

• Proposed approach: Virtual Cane—a mobile VR system using a smartphone mounted on a selfie stick, combined with spatial audio and vibration feedback, to simulate white cane techniques for BLV users.
• Novelty:
  1. Development of a dual-layer virtual object model for pseudo-haptic feedback.
  2. Integration of spatial audio and vibrotactile cues for object boundary detection and navigation.
  3. Use of consumer-grade devices (smartphone, headphones, selfie stick) for accessibility and portability.
  4. Evaluation of cognitive map formation and real-world transfer through user studies.
• Procedure and key techniques:
  • Smartphone tracks cane movement via AR Foundation.
  • Headphones provide spatial audio cues for object direction and proximity.
  • Dual-layer virtual objects simulate tactile resistance through auditory and vibration feedback.
  • Training modules teach virtual cane techniques, including constant contact, two-point touch, and shorelining.

Results

• Concrete findings:
  • Participants identified nearly all structural components (M=6.92/7) and objects (M=7.77/8) in virtual environments (VE).
  • Cognitive mapping performance in VE was comparable to real-world scenarios, with participants recalling most structural components (M=6.31/7) and objects (M=5.46/8).
  • Real-world transfer tasks showed accurate identification of structural components (M=6.31/7) and objects (M=5.38/9) based on VE-acquired mental maps.
• Advantage over baselines:
  • Faster exploration (0.26 minutes/m² vs. 0.70 minutes/m² in prior work).
  • Higher object identification accuracy (87.5% vs. 75% in prior studies).
  • Comparable or superior immersion and realism scores compared to existing methods.
• Experiments / evaluation:
  • Conducted with 13 blind adults (ages 23–72) using tasks in real-world, virtual, and mock-up environments.
  • Metrics included exploration time, cognitive map accuracy, and mental map transfer.
  • Feedback collected through interviews and questionnaires.
• Limitations and future work:
  • Limited participant pool and static environments.
  • Differences between virtual and real cane mechanics.
  • Need for evaluations in more complex, dynamic, and larger-scale environments.
  • Potential enhancements include mounting sensors on real canes, improving haptic fidelity, and supporting multi-level or crowded spaces.

Summary

This paper introduces Virtual Cane, a mobile VR system enabling BLV individuals to explore virtual environments using multimodal feedback on consumer devices. The system effectively supports cane-based navigation, cognitive mapping, and real-world transfer of spatial knowledge. A user study with 13 blind participants demonstrated high accuracy in identifying structures and objects, comparable to real-world exploration. While the system is accessible and portable, future work will address limitations in haptic fidelity, participant diversity, and environmental complexity. The approach holds promise for applications in O&M training, education, and entertainment.