LifeInsight: Design and Evaluation of an AI-Powered Assistive Wearable for Blind and Low Vision People Across Multiple Everyday Life Scenarios

Research Background and Issues

• Identified Issues or Challenges: Current assistive technologies (ATs) can support blind and low-vision individuals in certain daily activities, but most devices have limitations in specific scenarios and high interaction complexity, making them unsuitable for mobile contexts. This forces blind and low-vision users to rely on multiple devices, which are often underutilized. Key challenges revolve around multi-scenario needs such as object recognition, text reading, and device interface interaction.

• Importance: Intelligent assistive devices can effectively reduce the dependence of blind or low-vision individuals on others, enhance social participation and quality of life, and simplify the constraints of existing complex device scenarios.

• Research Motivation and Related Work: Existing studies have explored how blind individuals use mobile applications (e.g., Seeing AI and Be My Eyes) to solve visual problems or rely on remote workers for assistance. Transitioning to wearable devices, especially those integrated with AI, offers the potential to free users' hands and improve interaction efficiency, thereby enhancing user experience. However, designing multi-scenario general-purpose AI assistive devices and promoting their adoption remains an urgent challenge.

Solution

• Proposed Method or Solution: The authors designed an AI-driven wearable assistive device named LifeInsight, which includes a wearable camera, microphone, and single-click interaction interface, allowing users to pose visual questions via voice input. The device integrates multi-scenario Q&A functionality and provides specialized information feedback through goal-oriented visual recognition (e.g., distinguishing food cans or checking candle status).

• Innovations:

  1. Overcomes the limitations of traditional handheld devices by enabling hands-free interaction through wearable hardware.
  2. Utilizes multimodal large language models (OpenAI GPT-4 Vision API) to optimize visual problem-solving.
  3. Provides a general-purpose solution for multi-scenario needs, including object localization, status checking, text reading, and comparisons.

• Implementation Steps:

  1. Needs Collection: Using cultural probes to collect videos and interviews of blind and low-vision individuals' daily activities over a week to understand key challenges.
  2. Device Design and Implementation: Successfully developed the LifeInsight device, integrating language model APIs for real-time visual problem analysis and voice feedback.
  3. Device Evaluation: Conducted experiments to assess LifeInsight's performance across six daily scenarios, including object localization, text comprehension, and environmental navigation.

Research Outcomes

• Specific Outcomes:

  1. Delivered an AI-driven assistive device that significantly improves multi-scenario interaction, with evaluations showing accurate visual problem-solving in over 80% of scenarios.
  2. Users reported that LifeInsight substantially reduced friction caused by using multiple devices, especially compared to smartphone applications.

• Advantages: Compared to existing smartphone applications (e.g., Seeing AI), LifeInsight's wearable form enhances ease of use and significantly reduces interaction complexity through voice and single-click interfaces. Additionally, the device supports goal-oriented information output, minimizing irrelevant redundant information.

• Experimental or Evaluation Results:

  1. Users initiated an average of 23 visual queries, with 41.4% of queries being goal-oriented scenario needs.
  2. The device's System Usability Scale (SUS) score was 74.81, indicating high usability.
  3. Users highlighted the device's exceptional performance in determining candle status, achieving 100% accuracy.

• Limitations and Future Directions:

  1. The sensing range is constrained by the camera's field of view, occasionally leading to errors in navigation or object localization scenarios (e.g., handling targets outside the line of sight).
  2. Further optimization of privacy protection measures is needed to address potential bystander privacy concerns arising from continuous monitoring in daily use.
  3. Investigate how user customization or trainable AI features can enhance the model's understanding of user needs.

Future work could delve into the long-term behavioral changes and social acceptance of the device among blind and low-vision individuals, while exploring ways to enhance LifeInsight's privacy protection capabilities and improve AI accuracy in solving complex visual problems across diverse scenarios.