AI That Moves With You: A Review of Interactive Technologies Powered by Large Foundation Models for Mobility Impairment

Paper Title

AI That Moves With You: A Review of Interactive Technologies Powered by Large Foundation Models for Mobility Impairment

Publication Info

• Topic area: Application of large foundation models in assistive technologies for mobility impairments.
• Keywords: Foundation models, large language models, vision language models, mobility impairment, assistive technology, human-computer interaction, accessibility, robotics, wearables, navigation.

Background and Problem

• Problem / challenge: Traditional assistive technologies rely on rigid, narrow pipelines and lack flexibility, adaptability, and personalization. Current research on foundation models in assistive systems is fragmented, with limited focus on mobility impairments.
• Significance: Mobility impairments affect hundreds of millions globally, impacting independence, healthcare, and quality of life. Addressing these challenges with advanced AI systems can transform accessibility and autonomy for affected individuals.
• Motivation and related work: Prior work has advanced assistive technologies through mechanical systems, signal-driven devices, and adaptive platforms, but persistent gaps remain in usability, robustness, and real-world validation. Foundation models offer capabilities in multimodal reasoning and flexible interaction, but their integration into mobility-related assistive systems lacks systematic synthesis.

Solution

• Proposed approach: A scoping review of FM-enabled interactive systems for mobility impairments, analyzing methodologies, integration strategies, interaction paradigms, and challenges.
• Novelty:
  1. Systematic taxonomy of FM-enabled interactions for mobility impairments.
  2. Tabulated corpus with reproducible codebook capturing design patterns and evaluation methods.
  3. Forward-looking research agenda emphasizing robustness, personalization, safety, and equity.
• Procedure and key techniques:
  • Screening 6,249 records from five databases, yielding 26 papers.
  • Coding papers based on research contributions, evaluation methods, FM integration strategies, and problem domains.
  • Synthesizing findings into conceptual design spaces and identifying technical and ethical challenges.

Results

• Concrete findings:
  • 53.8% of systems target blind or low-vision users; 33.3% focus on information access and comprehension, 29.6% on navigation, 29.6% on health self-management, and only 7.4% on physical assistance.
  • Most systems rely on large language models (LLMs) or vision language models (VLMs), with limited multimodal integration.
  • Common FM roles include reasoner (59%), orchestrator (19%), accelerator (11%), and planner-actuator bridge (4%).
• Advantage over baselines:
  • Improved accuracy in sign language recognition, navigation, and GUI accessibility.
  • Enhanced user experience, including reduced workload, increased autonomy, and higher trust.
  • Expanded interaction modalities, such as conversational journaling and wearable visual querying.
• Experiments / evaluation:
  • Evaluation methods include system-focused tests, formative studies, and field deployments.
  • Metrics range from task success rates and latency to usability scales (e.g., SUS, NASA-TLX).
  • Small-scale participant groups dominate, with limited longitudinal studies.
• Limitations and future work:
  • Challenges include latency, compute constraints, multimodal integration fragility, reasoning limits, and dataset scarcity.
  • Ethical concerns include privacy risks, bystander exposure, reliability in high-stakes contexts, and equity gaps.
  • Future directions include participatory design, standardized benchmarks, cross-disability systems, and robust FM integration strategies.

Summary

This review synthesizes 26 studies on FM-enabled interactive technologies for mobility impairments, highlighting their potential to address long-standing challenges in accessibility and autonomy. Systems predominantly target blind or low-vision users and focus on information access, navigation, and health self-management. While FM roles such as reasoner and orchestrator enhance interaction quality, technical and ethical challenges persist, including latency, privacy, and equity concerns. The paper proposes a structured taxonomy and research agenda to guide future work, emphasizing participatory design, robust evaluation frameworks, and scalable FM integration. These insights aim to inspire inclusive and effective assistive systems within HCI and the CHI community.