LLM-based Embodied Conversational Agent for Reducing Foreign Language Speaking Anxiety in Social VR

Paper Title

LLM-based Embodied Conversational Agent for Reducing Foreign Language Speaking Anxiety in Social VR

Publication Info

• Topic area: Reducing foreign language speaking anxiety using LLM-based conversational agents in virtual reality.
• Keywords: Foreign language speaking anxiety, LLM-based agents, social VR, immersive role-play, multimodal interaction, language learning, VRChat, anxiety reduction, embodied conversational agents, adaptive feedback.

Background and Problem

• Problem / challenge: Foreign Language Speaking Anxiety (FLSA) hinders language acquisition and societal integration, with existing interventions often limited to classroom settings and lacking realistic, low-stakes speaking practice in dynamic social contexts.
• Significance: Addressing FLSA can improve learners' confidence, language proficiency, and ability to engage in real-world communication, which is critical for personal and professional growth.
• Motivation and related work: Prior work has explored VR and chatbots for anxiety reduction, but limitations include lack of embodiment, expressiveness, and adaptive interaction. LLM-based ECAs in VR remain underexplored for FLSA, particularly in terms of their long-term effects and design considerations.

Solution

• Proposed approach: ELLMA-T, an LLM-based embodied conversational agent in social VR, designed to reduce FLSA through immersive, adaptive role-play scenarios.
• Novelty:
  1. Mixed-methods evaluation of an LLM-based ECA for spoken English practice, focusing on FLSA reduction.
  2. Empirical insights into how context and agent design influence learner experience and affective outcomes.
  3. Design considerations for building adaptive, multimodal ECAs in immersive environments.
• Procedure and key techniques:
  • Developed three role-play scenarios (classroom, café, group interaction) in VRChat.
  • Enhanced agent with multimodal capabilities (e.g., vision-based contextual feedback) and adaptive communication strategies.
  • Conducted a five-day study with 20 participants, combining quantitative (SLSAS, interaction metrics) and qualitative (interviews) methods to evaluate FLSA reduction and user experience.

Results

• Concrete findings:
  • Significant reduction in FLSA scores from pre-study (M = 2.38, SD = 0.83) to post-study (M = 1.91, SD = 0.67), t(19) = 4.55, p < 0.001.
  • Anxiety reduction became statistically significant after Day 3.
  • Increased interaction duration (p = 0.027) and speaking duration per turn (p = 0.048) from Day 1 to Day 5.
• Advantage over baselines:
  • Multimodal and embodied features provided richer, more engaging interaction compared to prior non-embodied or text-based systems.
  • Participants reported greater confidence and perceived real-life transferability of skills.
• Experiments / evaluation:
  • Mixed-methods study with 20 participants (ages 18–54, varied proficiency levels).
  • Quantitative measures: SLSAS, CEFR-based proficiency analysis, interaction metrics.
  • Qualitative data: Thematic analysis of interviews on user experience, anxiety triggers, and agent feedback.
• Limitations and future work:
  • No control group to isolate effects of immersion or LLM adaptivity.
  • Short study duration (five days) limits long-term insights.
  • Technical challenges like latency and occasional out-of-context agent responses.
  • Future work should include longitudinal studies, diverse participant profiles, and refined evaluation frameworks.

Summary

This study demonstrates the potential of LLM-based embodied conversational agents (ECAs) in social VR to reduce foreign language speaking anxiety (FLSA). The ELLMA-T system, featuring multimodal interaction and adaptive feedback, significantly reduced FLSA within five days and showed promise for real-life skill transfer. Participants highlighted the agent's embodiment, verbal and non-verbal behaviors, and scenario realism as key factors contributing to reduced anxiety and increased engagement. While limitations such as short study duration and technical constraints remain, the findings offer valuable insights for designing scalable, adaptive language learning systems.