CoEmpaTeam: Enhancing Cognitive Empathy using LLM-based Avatars and Dynamic Role Play in Virtual Reality

Paper Title

CoEmpaTeam: Enhancing Cognitive Empathy using LLM-based Avatars and Dynamic Role Play in Virtual Reality

Publication Info

• Topic area: Cognitive empathy training using VR and LLM-driven avatars.
• Keywords: Cognitive empathy, virtual reality, large language models, role-switching, perspective-taking, avatar design, immersive training, HCI, empathy transfer, personality modeling.

Background and Problem

• Problem / challenge: Cognitive empathy, the ability to understand others' perspectives, is declining, and traditional training methods (e.g., classroom programs, online tutorials) lack scalability, relevance, and embodied experience. Existing VR systems do not fully integrate role-switching or distinct avatar personalities to enhance empathy training.
• Significance: Cognitive empathy is crucial for effective communication, reducing biases, and fostering social harmony. Developing scalable and effective training methods can address this societal need.
• Motivation and related work: Prior research highlights the potential of VR for immersive perspective-taking and the use of LLM-driven avatars for dynamic interaction. However, these elements have not been combined into a unified framework for empathy training. This paper builds on these insights to address the gap.

Solution

• Proposed approach: CoEmpaTeam, a VR-based system that uses LLM-driven avatars with distinct personalities and a role-switching mechanism to train cognitive empathy in a co-living scenario.
• Novelty:
  1. Integration of role-switching and LLM-driven avatars to foster multi-perspective social encounters.
  2. Validation of avatar personalities using the Big Five framework through self-assessment and human evaluation.
  3. Empirical evidence of cognitive empathy improvement and real-world transfer through a three-week study.
  4. Open-source release of the system for reproducibility and further research.
• Procedure and key techniques:
  • Participants engage in a co-living task with two avatars, alternating roles across three sessions.
  • Avatars embody distinct personalities based on the Big Five framework, validated through NEO-FFI-30 assessments and participant evaluations.
  • Role-switching encourages perspective-taking by requiring participants to adopt and negotiate from multiple viewpoints.
  • The system integrates LLMs for dialogue generation, emotion, and gesture alignment, enhancing avatar realism and interaction.

Results

• Concrete findings:
  • Significant increases in cognitive empathy: Perspective Taking (PT) scores improved from M = 3.36 to M = 3.70 (p = .007, d = 0.64), and Fantasy (FS) scores improved from M = 2.76 to M = 3.22 (p < .001, d = 0.86).
  • Participants reported applying empathy strategies in real-world interactions during a diary study.
• Advantage over baselines:
  • Role-switching and avatar personality consistency enabled deeper perspective-taking compared to traditional role-play or static VR scenarios.
  • The system provided scalable, consistent, and adaptive training without requiring multiple actors or participants.
• Experiments / evaluation:
  • Study 1: Avatar validation with 90 participants confirmed personality consistency through self-assessment (cosine similarity: Alice = 0.997, Benji = 0.862, Caden = 0.950) and human evaluation (correlations: Alice r = .93, Benji r = .87, Caden r = .89).
  • Study 2: Three-session training with 32 participants (22 completed all sessions) and a one-week diary study demonstrated cognitive empathy improvements and real-world transfer.
  • Measures included the Interpersonal Reactivity Index (IRI), Presence Questionnaire (PQ), and NASA-TLX for task load.
• Limitations and future work:
  • Homogeneous sample (young university students) limits generalizability.
  • Cultural influences on empathy training were not analyzed.
  • No non–role-switching baseline for comparison.
  • Occasional latency in avatar responses disrupted conversational flow.
  • Future work should explore long-term effects, cultural adaptivity, and expanded role scenarios.

Summary

CoEmpaTeam demonstrates the potential of VR-based role-switching with LLM-driven avatars to enhance cognitive empathy. The system validated distinct avatar personalities and showed significant improvements in perspective-taking and fantasy scores, with skills transferring to real-world contexts. By integrating structured tasks, relatable scenarios, and scalable design, this approach offers a promising framework for empathy training and broader soft-skill development. Future research should address cultural diversity, long-term effects, and technical refinements to expand applicability.