Let's Face It: Influence of Facial Expressions on Social Presence in Collaborative Virtual Reality

Title of the Paper

Let’s Face It: Influence of Facial Expressions on Social Presence in Collaborative Virtual Reality

Paper Information

• Subject Area: Enhancing social presence in Social Virtual Reality (VR) and the role of facial expressions
• Keywords: Virtual Reality, Collaboration, Social Presence, Facial Expressions, User Study

Research Background and Problem Statement

• Identified Problems or Challenges:

  1. Current collaborative virtual reality applications fail to provide a sense of social connection comparable to face-to-face interactions.
  2. Multimodal levels and nonverbal cues in virtual reality environments are limited.
  3. Existing studies rarely focus on the impact of facial expressions on social presence within virtual reality platforms or the role of task types in this context.

• Significance: With the growing prevalence of globalization and remote work, physical distances between individuals in personal and professional contexts are increasing. Social virtual reality can help bridge this gap and foster social connections, offering potential solutions for various industries and personal domains.

• Research Motivation and Related Work:

  • The authors highlight the critical role of facial expressions in real-life interpersonal communication, while research in virtual reality remains in its infancy.
  • Existing studies often focus on asymmetric setups (e.g., interactions between computer users and VR users), lacking research on authentic multi-user VR environments.
  • This study systematically explores how facial expressions influence social presence in social virtual reality using emerging facial tracking technologies.

Proposed Solution

• Proposed Approach:

  • Implement four types of facial expressions in virtual reality using recent facial tracking technologies (eye movement, mouth movement, a combination of both, and neutral expressions) and evaluate their impact on two collaborative tasks (verbal explanation and graphical explanation).
  • Conduct a controlled experiment (N=48) to observe the impact of facial expressions on social presence in social virtual reality.

• Innovations:

  • The first systematic study to investigate how facial expressions in social virtual reality interact with different collaborative tasks to enhance social presence.
  • Explores the role of dynamic facial expressions in a fully VR setting, addressing gaps in existing human-computer interaction research.

• Implementation Steps and Key Technologies:

  1. Utilize advanced VR equipment and tracking technologies to achieve real-time facial expression capture (e.g., Tobii eye tracking, VIVE mouth tracking).
  2. Design verbal or graphical explanation tasks to simulate real-world collaboration needs.
  3. Evaluate social presence under different conditions using psychological questionnaires and behavioral measurements (e.g., gaze duration, mouth movement changes).

Research Outcomes

• Specific Findings:

  • Facial expressions significantly enhanced participants' sense of co-presence, particularly the combination of eye and mouth movements during verbal tasks.
  • Graphical tasks had a smaller impact on facial expression rendering, but fully dynamic facial animations still contributed to the realism of VR tasks.

• Advantages of the Proposed Solution:

  • Compared to neutral expressions, dynamic facial expressions (especially mouth movements) significantly improved social presence.
  • Provided detailed insights into the interaction between VR task types and the impact of facial expressions, enabling developers to better enhance user experience.

• Experimental or Evaluation Results:

  • Verbal tasks generally exhibited higher levels of social presence.
  • Dynamic facial expressions (combined eye and mouth movements) increased participants' gaze duration towards each other and enhanced emotional understanding.

• Limitations and Future Directions:

  • Limitations: Participants' perception of facial expression changes was relatively limited; the tracking precision and variety of expressions were constrained by hardware capabilities.
  • Future Directions:
    • Expand the range of expressions (e.g., eyebrow movements, facial wrinkles).
    • Explore task design (e.g., incorporating whiteboard collaboration) to alter nonverbal cues.
    • Consider additional environmental factors influencing social presence, such as user relationships or VR application types.

Additional Design Recommendations

• Suggestions for Future VR Development:
  • Prioritize real-time rendering of mouth movements to enhance users' perception of others' emotions and efforts.
  • Use task types that focus on visual attention (e.g., verbal explanations) to amplify users' awareness of social activities.
  • Dynamically adjust the granularity of facial expression rendering based on hardware or network constraints.

Conclusion and Contributions

This study investigated the impact of facial expressions on social presence in virtual reality, demonstrating the significant potential of dynamic facial expressions to enhance collaborative experiences. Through a combination of quantitative and qualitative analyses, the authors proposed critical design recommendations, offering new perspectives and tools for the future development of social virtual reality.