Pedagogical Agents in Educational VR: An in the Wild Study

Title of the Paper

Pedagogical Agents in Educational VR: An in-the-Wild Study

Paper Information

• Field of Study: Educational technology, optimization of multimedia learning theory in virtual reality learning
• Keywords: immersive virtual reality, educational technology, learning, cognitive load, pedagogical agents, social presence

Research Background and Issues

• Identified Problems or Challenges

  1. Whether pedagogical agents in virtual learning environments truly enhance learning remains uncertain.
  2. Pedagogical agents may increase learners' cognitive load, interfering with learning efficiency.
  3. The applicability of multimedia theory from non-immersive traditional learning environments to immersive virtual reality (VR) is still debated.

• Significance of the Research During the COVID-19 pandemic, the demand for remote learning surged, making the use of pedagogical agents and VR technology critical for providing new methods in distance education. Building more efficient remote education models can help overcome the practical limitations of traditional face-to-face teaching.

• Motivation and Related Work

  1. Existing studies suggest immersive learning environments may enhance learning interest, but the outcomes do not necessarily improve learning performance.
  2. Social agency theory posits that human-like pedagogical agents and social cues in multimedia learning can promote deep learning by enhancing social presence, but this lacks sufficient empirical validation.
  3. Literature reviews and experimental results show inconsistencies, such as conflicting findings on the impact of pedagogical agents' visual realism on learning.

Solution

• Methods or Solutions

  1. Design a virtual museum focused on virology topics and test the impact of pedagogical agents on learning with 161 participants using Oculus Quest headsets at home.
  2. Employ a 2 × 2 + 1 design method, manipulating the visual realism and behavioral realism of pedagogical agents: high/low visual realism + high/low behavioral realism, along with a control group with voice-only narration.
  3. Evaluate learning outcomes objectively and subjective experiences, including knowledge tests (two types of questions: factual and conceptual), social presence, and self-perceived cognitive load.

• Innovations

  1. Combine social agency theory with immersive VR learning to study its theoretical adaptability and optimization principles in education.
  2. Investigate the impact of consistency between visual realism and behavioral realism on learning outcomes, addressing gaps in previous experiments.
  3. Collect data from participants' home environments, enhancing external ecological validity.

• Implementation Steps and Key Technologies

  1. Develop the VR environment (Unity 2020) and pedagogical agent models (from Microsoft Rocketbox library).
  2. Design learning content based on audio, visual slides, and animations (including knowledge about measles, Zika virus, COVID-19 transmission, and vaccines).
  3. Conduct user testing: pre-test and post-test knowledge assessments, combined with Likert scale evaluations of subjective learning experiences.
  4. Analyze data using ANOVA to statistically test variables such as knowledge gain and social presence.

Research Findings

• Specific Results

  1. Learning Outcomes:
     • Overall knowledge test scores increased significantly from 11.0 (out of 20) in the pre-test to 15.1 in the post-test, indicating substantial knowledge gain (Cohen’s d = 1.4).
     • In factual knowledge learning, the control group without pedagogical agents scored the highest; in conceptual knowledge learning, agents with inconsistent visual and behavioral realism performed best.
  2. Subjective Feedback:
     • Participants generally rated the VR learning experience highly (M = 3.9/5).
     • High visual realism in agents may trigger the "uncanny valley" effect, with some participants reporting discomfort with the agents.
  3. Enhanced Social Presence: Behavioral realism in pedagogical agents significantly improved social presence.

• Advantages Compared to Existing Solutions

  1. The experiment verified that factual knowledge learning is more prone to interference from pedagogical agents, while conceptual knowledge learning shows greater potential.
  2. Provided new insights into the consistency of visual and behavioral realism, contradicting previous claims about consistency effects.
  3. Demonstrated the feasibility of combining VR with autonomous and remote experiments.

• Experimental or Evaluation Results

  • High behavioral realism (e.g., natural movements, eye contact) significantly increased social presence but showed potential benefits only in conceptual knowledge learning.
  • High visual or behavioral realism in agents increased cognitive load, negatively impacting factual knowledge learning.

• Limitations and Future Directions

  1. Limitations:
     • The sample primarily consisted of experienced VR users, and the generalizability of the results requires further validation.
     • The study did not explore the effects of learning transfer (e.g., applying learned knowledge in new scenarios).
  2. Future Directions:
     • Further investigate the application of pedagogical agents in other educational environments and subject areas.
     • Explore the potential impact of agent gender and roles (e.g., motivator or procedural demonstrator).
     • Use eye-tracking to study learners' attention distribution, gaining deeper insights into the interference or facilitation effects of pedagogical agents.

Conclusion

This study demonstrates that the impact of adding pedagogical agents in VR on learning depends on the type of knowledge. Pedagogical agents may distract learners in factual knowledge learning but show certain positive effects in deep conceptual learning. These findings have profound implications for optimizing multimedia theory in immersive learning environments, especially in the context of the growing importance of remote education.