Eyes on the Finger: Investigating a Ring-Shaped Camera for Seamless Accessible Tactile Exploration

Paper Title

Eyes on the Palm: Investigating a Ring-Shaped Camera for Seamless Accessible Tactile Exploration

Publication Info

• Topic area: Assistive technologies for blind and low vision (BLV) individuals, focusing on tactile exploration augmented by wearable cameras.
• Keywords: Tactile exploration, blind and low vision (BLV), wearable devices, ring camera, bimanual strategies, object recognition, assistive technology, museum accessibility, YOLO, GPT.

Background and Problem

• Problem / challenge: Existing tactile exploration tools for BLV individuals often disrupt natural hand movements or impose additional layers of interaction, such as pointing or scanning gestures, which are not well-suited for seamless tactile exploration. Smartphone-based and head-mounted camera systems also fail to integrate effectively with bimanual tactile strategies.
• Significance: Tactile exploration is crucial for BLV individuals to understand objects and spaces, particularly in real-world environments like museums. Enhancing this process with non-disruptive, camera-based tools can improve accessibility and engagement.
• Motivation and related work: Prior work has explored tactile graphics, 3D models, and camera-based assistive technologies like smartphones and finger-worn devices. However, these solutions often require explicit gestures or disrupt tactile engagement. This study builds on these foundations to investigate a ring-shaped camera that preserves tactile exploration while enabling information inquiry.

Solution

• Proposed approach: A ring-shaped camera worn on the finger, designed to capture images from the palm side while allowing continuous tactile exploration. The system integrates object recognition, contextual explanations, and haptic feedback.
• Novelty:
  1. Investigates how a ring camera supports seamless tactile exploration compared to smartphones.
  2. Analyzes bimanual coordination across exploration, inquiry, and processing phases.
  3. Implements and evaluates a functional system combining YOLO-based object recognition, GPT-generated descriptions, and tactile feedback.
  4. Provides design implications for wearable camera systems in real-world settings.
• Procedure and key techniques:
  1. Conducted a formative study with 13 BLV participants to evaluate the wearability of the ring camera.
  2. Performed a Wizard-of-Oz (WoZ) study with 11 BLV participants in a science museum to compare the ring camera with smartphones.
  3. Developed a functional system integrating YOLOv8 for object detection, GPT-4o for contextual descriptions, and haptic feedback.
  4. Evaluated the system with 6 BLV participants in a depth-rich museum exhibit, focusing on accuracy, usability, and exploration strategies.

Results

• Concrete findings:
  • The ring camera allowed participants to hover for information while maintaining tactile contact, unlike smartphones, which required both hands to disengage from the exhibit.
  • In Study 1, participants identified 89.9% of items with the ring camera compared to 83.6% with smartphones. The ring camera also resulted in lower workload scores (mean Raw-TLX: 35 vs. 52 for smartphones).
  • In Study 2, the system achieved 88.3% YOLO detection accuracy and 73.6% GPT description accuracy, supporting comprehensive exploration of depth-rich exhibits.
• Advantage over baselines:
  • The ring camera preserved bimanual tactile strategies, enabling seamless transitions between exploration, inquiry, and processing.
  • Smartphones disrupted tactile engagement, leading to fragmented exploration and higher physical and mental effort.
• Experiments / evaluation:
  • Study 1: WoZ comparison of ring camera and smartphone with 11 BLV participants exploring four museum exhibits.
  • Study 2: Evaluation of the implemented ring camera system with 6 BLV participants in a depth-rich exhibit, focusing on usability, accuracy, and exploration strategies.
  • Metrics included item identification rates, Raw-TLX workload scores, usability ratings, and qualitative feedback.
• Limitations and future work:
  • Limited to 17 BLV participants and a single thematic domain (museum exhibits).
  • Recognition accuracy was constrained by training datasets collected from sighted users, which did not fully capture BLV participants’ hand movements.
  • Future work should involve larger, more diverse participant groups, co-designed datasets, and long-term deployments to assess real-world applicability.

Summary

This study investigates a ring-shaped camera designed to support seamless tactile exploration for BLV individuals. Compared to smartphones, the ring camera enabled participants to hover for information while maintaining tactile contact, reducing workload and enhancing usability. A functional system integrating YOLO-based object recognition, GPT-generated descriptions, and haptic feedback was developed and evaluated in a depth-rich museum setting. Results demonstrated the system’s potential to augment tactile exploration without disrupting natural hand movements. Future research should address recognition accuracy, dataset diversity, and long-term usability to further refine this technology for broader applications.