The Impact of Observer Presence on Trainees' Mental States and Performance in Remote Military Training with Virtual Humans in Mixed Reality Environment

Research Background and Issues

What problems or challenges did the authors identify?

The authors pointed out that in remote military training, training effectiveness may decline due to insufficient attention and engagement from trainees. Traditionally, to compensate for the lack of in-person interaction in remote training, on-site observers are often assigned to supervise and evaluate trainees' performance. However, the presence of on-site observers may cause negative psychological effects, such as increased stress and cognitive load for trainees, especially during complex or unfamiliar tasks.

Why is this issue important?

Efficiently developing soldiers' execution capabilities and psychological adaptability is critical in military training, particularly when facing complex or novel training tasks. Excessive psychological stress may impair trainees' performance, while a lack of supervision could lead to distraction or reduced engagement. Therefore, balancing the need for supervision with minimizing psychological burden is particularly important.

Research Motivation and Related Work

Advancements in technology have made mixed reality (MR) and holographic remote education possible, providing immersive experiences for training scenario simulations. Although virtual humans (e.g., avatars) have been widely used in military simulations, their effectiveness as substitutes for on-site observers has not been thoroughly studied. The authors aim to explore whether virtual observers can achieve training outcomes similar to those of human observers while reducing psychological stress.

Solution

What methods or solutions did the authors propose?

The study tested the impact of observers by comparing the following three conditions:

1. Training without an observer (NO).
2. Training with an on-site human observer (RO).
3. Training with a virtual observer (VO).

The virtual avatar was remotely controlled in real time by trainers to simulate interactions with the task environment. Additionally, the study designed virtual observers whose visual and behavioral characteristics matched those of real observers but incorporated virtual elements through mixed reality technology.

What are the innovative aspects of this solution?

1. Mixed Reality Environment: The use of MR technology provided a realistic training environment, enhancing presence and immersion.
2. Application of Virtual Humans: Replacing traditional voice or text commands with real-time controlled virtual avatars improved collaboration and task engagement.
3. Testing Virtual Observers: The study explored the feasibility of virtual observers as substitutes for real observers, offering a potential solution to reduce psychological stress.

What are the implementation steps? What key technologies were used?

1. Experimental Design: A comparison of three conditions (NO, RO, VO) based on a single independent variable (presence and type of observer).
2. Equipment and Platform: Meta Quest 3 devices and an MR-based training platform were used to provide dynamic training content and guide trainees through tasks such as weapon selection and target shooting.
3. Measurement Metrics: Quantitative measures, including emotional change questionnaires, stress levels (SUDS), task workload (NASA-TLX), and presence questionnaires (IPQ), were used to evaluate the impact of observer presence on psychological states and task performance.
4. User Behavior Analysis: Trainees' avoidance behavior towards virtual avatars and task completion times were recorded to assess behavioral patterns.

Research Findings

What specific findings were obtained?

1. Changes in Psychological State:

   • The presence of on-site observers (RO) significantly increased trainees' stress and task workload (e.g., mental and temporal demands).
   • The impact of virtual observers (VO) was intermediate between real observers (RO) and no observers (NO), causing moderate psychological stress.

2. Task Performance:

   • Although the presence of observers affected trainees' psychological states, training task performance (e.g., shooting balloons and responding to tactical signals) was not significantly impacted.
   • The immersion of MR content and the effective guidance of virtual avatars likely mitigated the negative effects of observers on task performance.

3. Comparison with Existing Solutions:

   • Virtual observers performed better than real observers in maintaining training discipline and stabilizing trainees' psychological states, while avoiding the disruption of virtual environments by real-world elements (e.g., "presence breaks").
   • The absence of observers (NO) significantly reduced trainees' mental demands but may not be suitable for military training requiring strict discipline.

4. Expert Feedback:

   • Military experts believed that virtual avatars could effectively enhance training, especially in environments where physical simulations are impractical (e.g., high-risk operations).
   • However, aspects such as simulated tactile feedback (e.g., weapon weight and recoil) and emotional feedback capabilities require further improvement.

What were the experimental or evaluation results?

1. Psychological Stress:
   • SUDS scores showed that the RO group's psychological stress was significantly higher than that of the NO group. NASA-TLX task workload tests indicated that mental demand and frustration were highest in the RO group.
2. Training Quality:
   • Training engagement, spatial presence, and system usability scores were roughly equal across all groups, indicating that MR content and virtual avatars had a positive impact on overall training outcomes.
3. Avoidance Behavior:
   • Trainees in all three groups exhibited avoidance behavior towards virtual characters, suggesting that virtual avatars could evoke a certain level of social presence regardless of observer presence.

Limitations and Future Directions

1. This study did not directly compare the differences with traditional training environments (e.g., in-person and fully immersive VR), and future research should explore the combined impact of different environments on training outcomes.
2. Only male participants were recruited (due to South Korea's mandatory military service), which may limit the generalizability of the results.
3. The experimental setup restricted trainees' ability to freely explore content, differing from the autonomy soldiers might have in real-life scenarios.
4. Future research is recommended to further develop the emotional interaction capabilities of virtual avatars, such as real-time interpretation of trainees' emotions and providing encouragement to enhance training effectiveness in specific contexts.

Overall, this study provides design and practical guidance for military and other remote training fields (e.g., medical, technical education): using virtual observers and mixed reality technology can maintain high task engagement and training effectiveness while reducing psychological stress.