Engaged and Affective Virtual Agents: Their Impact on Social Presence, Trustworthiness, and Decision-Making in the Group Discussion

Document Title

"Engaged and Affective Virtual Agents: Their Impact on Social Presence, Trustworthiness, and Decision-Making in the Group Discussion"

Document Information

• Subject Area: Behavior design of virtual agents (VA) and their impact on group decision-making processes and user perception
• Keywords: Virtual agents, affective computing, engagement, emotional suppression, group decision-making

Research Background and Issues

• Problems or Challenges:

  • How do engagement and emotional behaviors (expressive or suppressed) of virtual agents in group task settings influence user perception and behavior?
  • How to design virtual agent behaviors to enhance the effectiveness of group collaboration and decision-making?
  • Emotional expression may lead to negative effects in certain scenarios, such as impacting social presence and trustworthiness.

• Significance:

  • With the expanding use of virtual agents in fields such as education, remote meetings, and healthcare, the alignment of their behavior design with task requirements directly affects user experience and interaction outcomes.
  • In group environments, task decision-making requires balancing task outcomes with social harmony. Studying these influencing factors can guide the functional design of virtual agents in organizational contexts.

• Research Motivation and Related Work:

  • Existing studies show that the visual and emotional cues of virtual agents significantly influence individual user interactions, but most focus on dyadic interaction.
  • Group settings (e.g., many-to-many interactions) remain underexplored, and emotional and engagement behaviors may have entirely different impacts on user group attention distribution and decision-making.

Solution

• Methods or Solution:

  • A "2×2" experimental design was proposed: constructing four types of virtual agent behaviors based on "engagement behavior" (Engaged [E] vs. Non-Engaged [NE]) and "emotional behavior" (Affective [A] vs. Non-Affective [NA]).
  • Through human-computer interaction experiments, the process of virtual agents interacting with two human participants in a group decision-making task was simulated.

• Innovations:

  • Explored the interaction dynamics of virtual agent engagement and emotional behaviors in multi-user (group) contexts.
  • Considered the actual impact of non-verbal behaviors (gaze, gestures) and emotional expressions (facial expressions, vocal tone, etc.) comprehensively, assessing their effects on social presence, trustworthiness, and attention distribution.

• Implementation Steps or Key Techniques:

  1. Virtual agents provided interactive behaviors (e.g., demonstrating "listening" or "speaking" behaviors through head turns and gaze coordination).
  2. Predefined emotional expressions, gestures, and vocal tones were used to simulate agent reactions in positive or negative emotional states.
  3. Eye-tracking technology was employed to capture participants' attention distribution during the discussion phase.
  4. Post-experiment questionnaire data were analyzed to assess users' subjective perceptions of social presence and trustworthiness.

Research Findings

• Specific Findings:

  • Engagement significantly enhanced users' attention allocation to the virtual agent but had limited impact on improving subjective perceptions (e.g., trust or social influence).
  • Affective behavior had a negative impact on users' subjective perceptions, particularly in terms of trust and social presence, but may influence decision acceptance in certain decision-making scenarios.

• Advantages Over Existing Solutions:

  • Provided a functional distinction between emotional and engagement behaviors in group decision-making and interaction.
  • Offered design guidance for virtual agents in more complex organizational environments, such as roles as facilitators in collaboration or mediators in decision-making.

• Experimental or Evaluation Results:

  • Attention Allocation: Users significantly allocated more attention to virtual agents exhibiting engaged behaviors (e.g., gaze and nodding). Non-engaged (NE) virtual agents failed to attract sufficient attention.
  • Group Task Performance: Agents with engaged behaviors (E-A, E-NA) better facilitated group collaboration and improved problem-solving accuracy.
  • Subjective Perception: Non-affective agents exhibiting engagement (E-NA) were considered the most ideal team members, while non-engaged agents with affective behaviors (NE-A) were rated as the least favorable.

• Limitations and Future Directions:

  • Limitations: The current experiment only tested interactions within three-person groups, which may not fully reflect the dynamics of more complex organizational environments.
  • Future Directions:
    1. Expand group size and diversity to explore broader multi-user interaction contexts.
    2. Enhance the dynamic adaptability of virtual agent emotional expressions to better align with users' real-time social environments.
    3. Conduct in-depth studies on the impact of virtual agent behaviors on long-term group relationships (e.g., trust accumulation).