Designing Emotion Feedback for Embodied Virtual Agents: A Continuous Emotional Intensity Model

Paper Title

Designing Emotion Feedback for Embodied Virtual Agents: A Continuous Emotional Intensity Model

Publication Info

• Topic area: Emotional feedback optimization in embodied virtual agents (EVAs)
• Keywords: Embodied virtual agents, emotion feedback, emotional intensity, empathy, synchronization, expectation, alignment model, multimodal interaction, user likability, adaptive systems

Background and Problem

• Problem / challenge: Prior research has focused on selecting emotion categories for EVAs but has neglected the question of emotional intensity, which is critical for optimizing user likability.
• Significance: Understanding and optimizing emotional intensity feedback can enhance human-agent interaction quality, particularly in personal companionship applications.
• Motivation and related work: Fixed-intensity and adaptive-intensity rules (e.g., synchronization) have been explored, but they fail to consistently meet user preferences. Emotional intensity feedback remains underexplored as an independent variable, leaving gaps in understanding user preferences and mechanisms.

Solution

• Proposed approach: Continuous Emotional Intensity Alignment Model (EAM), which dynamically adjusts EVA emotional intensity based on user preferences and states.
• Novelty:
  1. Identification of psychological mechanisms (empathy, synchronization, expectation) driving user preferences for EVA emotional intensity.
  2. Development of a quantitative alignment model for emotion feedback intensity.
  3. Validation of the model in both facial-only and multimodal (facial + voice) conditions.
  4. Comparison with baseline models (fixed-intensity, synchronization, empathy, and expectation).
• Procedure and key techniques:
  • Experiment 1: Induced happiness and sadness in participants, measured preferred EVA emotional intensity, and developed the alignment model.
  • Experiment 2: Validated the model using facial expressions only, comparing it with baseline models.
  • Experiment 3: Extended validation to multimodal conditions (facial expressions + voice), including qualitative interviews.

Results

• Concrete findings:
  • Positive states: Preferred EVA emotional intensity closely aligned with empathy mechanisms.
  • Negative states: Preferences involved a hybrid mechanism, deviating from empathy, synchronization, and expectation.
  • Alignment model outperformed fixed-intensity, synchronization, empathy, and expectation models in both facial-only and multimodal conditions.
• Advantage over baselines:
  • Alignment model ranked first in preference (34.0% in facial-only; 32.1% in multimodal) and achieved the highest liking and willingness-to-interact ratings.
  • Fixed-intensity model performed worst, with 55.5% last-place rankings in facial-only and 62.4% in multimodal conditions.
• Experiments / evaluation:
  • Experiment 1: 80 participants (47 valid cases), induced emotions via videos, evaluated 11 EVA facial expressions.
  • Experiment 2: 35 participants, ranked five models using facial expressions.
  • Experiment 3: 65 participants, ranked five models using facial expressions + voice, with post-experiment interviews.
• Limitations and future work:
  • Limited to happiness and sadness; future work should explore other emotions like anger and fear.
  • Relied on self-report measures; combining physiological and behavioral metrics is recommended.
  • Applicability in unconstrained voice or text-based interactions remains untested.

Summary

This study developed and validated a Continuous Emotional Intensity Alignment Model (EAM) for embodied virtual agents (EVAs), addressing gaps in emotional intensity feedback optimization. Through experiments, the model demonstrated superiority over fixed-intensity, synchronization, empathy, and expectation models in both facial-only and multimodal conditions. Positive emotional states favored empathic feedback, while negative states required a hybrid approach balancing empathy, synchronization, and expectation. The findings offer actionable guidelines for designing adaptive EVAs in personal companionship applications, with potential extensions to broader emotional categories and interaction modalities.