"It Brought the Model to Life": Exploring the Embodiment of Multimodal I3Ms for People who are Blind or have Low Vision

Research Background and Issues

Issues and Challenges

• The authors identified that blind or visually impaired individuals often lack accessible access to visual educational content and navigation maps.
• Traditional tactile graphics and labels (e.g., Braille) are limited in spatial size and content capacity, which affects information comprehension and practicality.
• Although 3D printed models, especially interactive 3D printed models with audio labels (I3Ms), can address some of these limitations, traditional audio labels provide preset and limited information.
• Existing research suggests that empowering machines with human-like behaviors (i.e., "empowerment") can make them more vivid and socially present, thereby enhancing user engagement and trust. However, nearly all studies focus on sighted individuals, with little research on the impact on blind users.

Significance

• Designing interactive, empowered 3D printed models for blind or visually impaired individuals can provide vivid and personalized learning experiences, helping them better understand complex concepts.
• In educational and museum settings, such designs can significantly improve the usability and user experience of these models.

Research Motivation and Related Work

• With advancements in intelligent agents and multimodal technologies (including haptic feedback and voice interaction), the authors believe that empowered I3Ms could enhance engagement and trust among blind users.
• The research motivation lies in exploring whether design approaches can make I3Ms more vivid and socially present, while encouraging blind users to interact more deeply with these models.

Solution

Methods and Design

• The authors proposed a set of empowerment design factors (including haptic feedback, personalized voice, etc.) and designed two multimodal interactive 3D printed models: the Egyptian Pyramid and the Saturn V Rocket.
• The I3Ms were configured in two states:
  • High Empowerment Mode (HEM): Incorporates more empowerment design elements, such as personalized voice, haptic feedback, first-person narration, etc.
  • Low Empowerment Mode (LEM): Simpler design without empowerment enhancements.

Innovations

• Empowerment from a non-visual perspective: Moving away from reliance on visual empowerment, focusing instead on tactile feedback, localized voice, and other elements more meaningful to blind users.
• Combining multimodal technologies (e.g., tactile interaction, voice dialogue systems) with empowerment design, expanding the application scenarios of I3Ms and enhancing the learning experience between artificial agents and users.

Implementation Steps and Key Technologies

1. Model Implementation:
   • Interaction between the model and components was achieved using Raspberry Pi and low-cost electronic components.
   • Google Cloud Speech API was used for voice input and output, integrated with ChatGPT for external searches.
2. Empowerment Design Factors:
   • Mastered five key empowerment factors in design, such as haptic feedback and first-person narration.
   • Continuously optimized non-visual interaction experiences, such as enhancing the model's "presence" through embedded local audio and haptic technologies.
3. Experimental Design:
   • A controlled study was conducted with 12 blind participants, using quantitative surveys (e.g., Godspeed Questionnaire, User Engagement Scale) and subjective feedback to evaluate the models' performance in terms of vividness, engagement, and trust.

Research Outcomes

Specific Findings

• Experiments demonstrated that the high empowerment mode (HEM) I3Ms were perceived by participants as more vivid and socially present, significantly enhancing user engagement.
• Compared to the LEM models, the HEM models resulted in longer interaction times and more interactive behaviors, indicating a user preference for high empowerment design.

Advantages

• The HEM mode effectively enhanced participants' immersion and enthusiasm when using I3Ms, supporting the authors' hypotheses on empowerment design (H#1 and H#2).
• In educational contexts, empowered I3Ms provided blind users with more engaging learning experiences, leveraging multimodal technologies to facilitate more efficient content delivery.

Limitations and Future Directions

• Limitations:

  • Results on model trustworthiness were ambiguous, as participants focused more on the accuracy of output information rather than interactive behaviors when evaluating trust.
  • The small sample size (12 participants) limited the generalizability of the results; further studies with larger samples are needed to confirm findings.
  • Some empowerment factors (e.g., small dialogues) were underutilized in the experiment, failing to fully demonstrate their potential.

• Future Directions:

  • Explore additional implementations of empowerment design factors, such as model textures, sizes, and embedded speakers.
  • Extend research to real-world application scenarios, such as long-term deployment in schools or museums, to observe user behavior and model acceptance over time.
  • Simulate higher levels of model personalization, allowing users to adjust empowerment parameters based on individual needs.
  • Consider integrating generative AI to further enhance the interactivity of I3Ms, providing richer and more natural experiences.

Through the analysis of these findings and limitations, the study provides clear directions for future empowered I3M designs while expanding the research domain of interactive design for blind users.