RASSAR: Room Accessibility and Safety Scanning in Augmented Reality

Document Title

RASSAR: Room Accessibility and Safety Scanning in Augmented Reality

Document Information

• Subject Area: Human-Computer Interaction, Augmented Reality Technology, Assistive Technology, Indoor Space Accessibility and Safety Auditing
• Keywords: Augmented Reality, Indoor Accessibility Auditing, Computer Vision, Data Visualization, Hazard Detection, Human-Computer Interaction

Research Background and Problem

• Identified Problem: Globally, residential accessibility is insufficient. For example, 90% of housing in the United States is inaccessible to people with disabilities, while 98% of newly built private homes in the UK fail to meet the needs of wheelchair users. Traditional indoor accessibility and safety audits require professionals to perform manual measurements, which are inefficient and costly.
• Importance: The accessibility and safety of indoor spaces are closely tied to the quality of life of residents, especially for groups such as people with disabilities, the elderly, families, and children. However, accessibility issues in housing are widespread, and existing assessment tools are neither convenient nor capable of addressing personalized needs.
• Research Motivation and Related Work: Researchers have developed several tools and methods for indoor accessibility auditing, such as checklist-based assessment tools (e.g., HSSAT). However, these methods fail to fully leverage modern computer vision and augmented reality technologies. Additionally, existing work has primarily focused on outdoor environments, with limited research on indoor spaces.

Solution

• Proposed Method: Development of an augmented reality-based mobile application, RASSAR, for semi-automatic identification, localization, and visualization of indoor accessibility and safety issues.
  • Utilizes LiDAR sensors and real-time computer vision technology to scan and reconstruct indoor spaces.
  • Categorizes indoor components into four problem types: object dimensions, object placement, hazardous items, and missing assistive devices.
  • Provides real-time AR visualization and interactive 3D summary reports.
• Innovations:
  • Combines LiDAR scanning with a customized computer vision model (YOLOV5) to achieve high-precision real-time indoor space reconstruction and issue detection.
  • Offers personalized scanning and auditing rules for different user groups (e.g., wheelchair users, individuals with low vision).
  • Supports interactive validation and customization through JSON-defined auditing rules.
• Implementation Steps and Key Technologies:
  1. Utilizes the Apple RoomPlan API, combining LiDAR and RGB data for real-time indoor space reconstruction.
  2. Customizes and trains the YOLOV5 model for detecting small indoor items (e.g., medications, knives).
  3. Encodes auditing rules via JSON to enable personalization and extensibility.
  4. Provides real-time AR visualization and interactive 3D summary models, with support for voice assistance.

Research Results

• Specific Outcomes:
  • The RASSAR tool can accurately and efficiently audit indoor spaces for accessibility and safety, detecting 20 types of issues (e.g., switch height, missing grab bars, presence of hazardous items).
  • Achieved average precision and recall rates of 0.86/0.83, with user-performed scans yielding precision and recall rates of 0.79/0.73.
  • Scanning speed is 3.5 times faster than manual auditing (average scanning time: 99.9 seconds).
• Advantages Over Existing Solutions:
  • Simplifies the traditional manual auditing process and precisely identifies indoor accessibility issues.
  • Eliminates the cost of data collection relying on specialized equipment, enabling operation via smartphones.
  • Provides enhanced personalized auditing functionality, addressing the unique needs of diverse user groups.
• Experimental and Evaluation Results:
  • Technical performance was evaluated in 10 real residential spaces, confirming the tool's accuracy and consistency.
  • User studies showed RASSAR is easy to use, with users expressing approval of its detection performance and issue categorization (average rating: 6/7).
• Limitations and Future Directions:
  • Current model accuracy for detecting small items needs improvement (e.g., confusion between knives and medications).
  • Expand the range of supported auditing issues, such as stairs and entrance facilities.
  • Offer finer-grained personalization and user customization features, such as custom auditing rules via user interfaces.
  • Explore additional application scenarios, such as accessibility auditing in public spaces and obstacle identification in space rental services.

Additional Information

• Data and Code Sharing: RASSAR's detection model, training dataset, and code are open-sourced on GitHub (link).
• Potential Application Scenarios:
  • Pre-assessment of accessibility at visited locations (e.g., hotels, Airbnb).
  • Assisting home renovations and space modifications to accommodate life changes (e.g., injuries, elder care).
  • Providing remote auditing support for occupational therapists (OTs).