Bringing Patient Mannequins to Life: 3D Projection Enhances Nursing Simulation

Title of the Paper

Breathing Life into Nursing Simulation: 3D Projection-Enhanced Nursing Simulation

Bibliographic Information

• Field of Study: Integration of interactive nursing education and augmented reality technology
• Keywords: Nursing simulation, mixed augmented reality, interactive models, 3D projection, dynamic facial expressions, educational technology, nursing education, non-verbal communication, facial symptom recognition, mannequin simulation system

Research Background and Problem

• Problems and Challenges: Current patient models (mannequins) used in nursing simulations feature static facial expressions, lacking dynamism and realism. This limits nursing students' interaction and learning outcomes during simulations, particularly when attention to patients' non-verbal cues (e.g., facial symptoms) is critical.
• Significance: Facial expressions play a crucial role in nursing practice, such as helping nurses identify patients' pain levels, mental states, and key symptoms of diseases (e.g., facial drooping). This skill is essential for improving the quality of actual nursing care.
• Research Motivation: There is limited research in the literature on applying dynamic facial expressions to nursing simulations. The high cost, teaching limitations, and insufficient impact on students' perception and experience in current simulation technologies indicate the need for a new low-cost solution.
• Related Work: Existing augmented reality (AR) technologies focus on visualizing internal anatomical structures or providing operational guidance, with little emphasis on enhancing dynamic facial expressions. Some studies have explored virtual reality (VR) or mixed assistant systems, but these face challenges such as bulky equipment and difficulty integrating with full-body models.

Solution

• Proposed Method: Development of a 3D projection-based augmented reality system capable of overlaying dynamic and realistic facial expressions onto a nursing mannequin.
• Innovations:
  1. The system captures the nursing instructor's facial expressions in real-time and projects them onto a 3D-printed facial mask on the patient mannequin.
  2. Provides a low-cost solution using simple hardware (projector, camera, and mask) to achieve dynamic facial expression display.
  3. Allows instructors to manually adjust abnormal facial expressions (e.g., stroke-induced facial drooping) to enhance the realism of simulation scenarios.
• Implementation Steps:
  1. Use photogrammetry to scan the mannequin's face and 3D print a mask to ensure projection alignment.
  2. Employ facial capture software (Faceware) to capture the instructor's facial expressions in real-time.
  3. Use the Unity engine to adapt and project the facial expressions.
  4. Integrate the system into a stroke nursing simulation scenario, completing the setup with the mask, projector, and control panel adjustments.
• Key Technologies:
  • Photogrammetry: For 3D scanning.
  • Facial capture technology (32 facial marker points).
  • Facial Action Coding System (FACS): To generate facial expression animations.

Research Outcomes

• Specific Results:
  • Patient mannequins with dynamic facial expressions significantly attracted more visual attention from students.
  • Students improved their ability to recognize abnormalities, identifying critical expressions like facial drooping more quickly in stroke scenarios.
  • Surveys indicated that the system enhanced interactivity compared to traditional static models and was reported to be highly beneficial for accurate assessment and nursing decision-making.
• Experimental and Evaluation Results:
  1. Visual Attention Distribution: Eye-tracking revealed that students in the experimental group spent significantly more time focusing on the mannequin's face (37.53% of total time) compared to the control group (18.28%).
  2. Clinical Performance: The experimental group significantly reduced the time required to complete critical assessment stages, achieving a 100% accuracy rate in stroke identification, whereas two groups in the control condition failed to identify the stroke.
  3. Learning Experience: Survey results showed that students in the experimental group believed the dynamic facial expression model significantly improved interactivity and assessment effectiveness.
• Advantages Over Existing Solutions:
  • Lightweight system: No need for students to wear bulky equipment.
  • Low cost: Requires only 3D-printed components and standard cameras.
  • Broad applicability: Compatible with most existing full-body patient mannequins.
• Limitations and Future Directions:
  • Limitations:
    1. The position of the mannequin and projector must remain fixed.
    2. Strict control of lighting conditions is required to avoid interference with facial expression display.
    3. The animation style is cartoonish and needs improved realism.
    4. The current system requires manual control, increasing the instructor's workload.
  • Future Directions:
    1. Develop projection adjustments that automatically adapt to mannequin movement.
    2. Utilize AI technology to achieve system intelligence and automation, reducing manual intervention.
    3. Enhance the realism of facial animations.
    4. Expand the system's applicability to other nursing simulation scenarios.
    5. Conduct long-term studies to evaluate the system's impact on students' practical nursing skills.

Conclusion

By integrating 3D projection and dynamic facial capture technologies, this study demonstrates significant improvements in nursing simulation models. The system not only enhances students' learning outcomes and experiences during simulations but also provides a viable direction for the future development of nursing education technologies.