Should the AI Speak First? Evaluating Proactive vs. Reactive Facilitation in Mixed-Reality Medical Training

Paper Title

Should the AI Speak First? Evaluating Proactive vs. Reactive Facilitation in Mixed-Reality Medical Training

Publication Info

• Topic area: Interaction design for AI facilitators in immersive medical training.
• Keywords: AI facilitation, mixed reality, proactive AI, reactive AI, medical simulation, learner autonomy, cognitive load, interaction timing, procedural training, HCI.

Background and Problem

• Problem / challenge: Determining the optimal level of AI proactivity in immersive medical training environments remains unresolved. Proactive AI may help learners stay on track but risks being intrusive, while reactive AI may preserve autonomy but leave learners unsupported during critical moments.
• Significance: Understanding how AI facilitation affects learning outcomes and experiences is crucial for designing effective, scalable training systems in high-stakes fields like medicine.
• Motivation and related work: Prior research shows that AI-enhanced XR systems improve learning outcomes compared to XR alone, but little is known about how proactive vs. reactive AI facilitation influences learner behavior and perceptions in embodied, immersive training contexts. This study addresses this gap by comparing proactive and reactive AI facilitators in a mixed-reality lumbar puncture simulator.

Solution

• Proposed approach: A comparative study of two AI facilitation modes—Proactive AI (initiates guidance) and Reactive AI (responds only to learner prompts)—within an XR-based lumbar puncture training system.
• Novelty:
  1. Empirical analysis of how AI initiative, timing, and situational relevance shape learner experiences in immersive procedural training.
  2. Mixed-methods evaluation combining behavioral video coding, quantitative measures, and stimulated recall interviews.
  3. Development of a boundary framework (Initiative, Temporal, Situational) for calibrating AI proactivity in immersive training systems.
• Procedure and key techniques:
  • Developed an XR lumbar puncture simulator with AI facilitation using the Apple Vision Pro headset.
  • Designed proactive and reactive AI facilitators powered by a Gemini Flash language model.
  • Conducted a between-subjects study with 22 medical students, analyzing learning outcomes, interaction patterns, and learner perceptions.

Results

• Concrete findings:
  • No significant differences in learning outcomes (knowledge gain, self-efficacy, task performance) between proactive and reactive AI conditions.
  • Both groups showed measurable improvements in procedural knowledge and confidence after training.
  • Learners in the reactive condition initiated slightly more interactions, but differences were not statistically significant.
• Advantage over baselines: Both AI facilitation modes supported positive learning outcomes, but neither outperformed the other in terms of objective metrics.
• Experiments / evaluation:
  • Participants (n=22) were randomly assigned to proactive (n=11) or reactive (n=11) AI conditions.
  • Data collection included pre- and post-training surveys, expert-graded performance assessments, and video-recorded sessions.
  • Behavioral and thematic coding revealed nuanced differences in learner-AI interaction patterns and perceptions.
• Limitations and future work:
  • Study focused on a single procedural skill (lumbar puncture) and a single session per learner.
  • Proactive triggers were developmental and not fully optimized.
  • Small sample size limits detection of small quantitative differences.
  • Future work should explore adaptive, real-time calibration of AI proactivity and evaluate generalizability across domains and learner populations.

Summary

This study compared proactive and reactive AI facilitators in an XR-based lumbar puncture simulator to examine how AI initiative affects learning outcomes and experiences. While no significant differences in learning outcomes were observed, behavioral and qualitative analyses revealed distinct interaction patterns and learner preferences. A boundary framework (Initiative, Temporal, Situational) was proposed to guide the design of AI facilitators in immersive training systems. These findings provide actionable insights for calibrating AI proactivity to balance learner autonomy, task rhythm, and situational relevance, with implications for broader applications in immersive learning environments.