Towards Leveraging AI-based Moderation to Address Emergent Harassment in Social Virtual Reality

Document Title

Towards Leveraging AI-based Moderation to Address Emergent Harassment in Social Virtual Reality

Document Information

• Subject Areas: Human-Computer Interaction (HCI), Artificial Intelligence (AI) Applications, Social Virtual Reality (Social VR), Online Content Moderation
• Keywords: Artificial Intelligence, Content Moderation, Online Harassment, Social Virtual Reality, User Collaboration, Human-AI Collaboration, Safety

Research Background and Issues

• Existing Problems and Challenges:

  • Harassment in Social Virtual Reality (Social VR) differs from traditional online environments, including issues like virtual sexual assault or voice harassment.
  • Current online harassment governance methods (e.g., manual or community-driven content moderation mechanisms) are limited in effectiveness within this context.
  • There is a lack of industry consensus on addressing harassment in Social VR, and emerging AI moderation mechanisms have not been effectively implemented.

• Importance of the Research Problem:

  • The immersive interaction and real-time voice communication features of Social VR make harassment issues more complex and pronounced.
  • Current platform moderation approaches have vague definitions of harassment and lack transparency in enforcement, leaving users and communities feeling unsafe.

• Research Motivation and Related Work:

  • The authors conducted a study with 39 Social VR users to explore their perceptions of AI moderation and how new moderation mechanisms can address the shortcomings of existing methods.
  • Building on existing literature on online harassment governance, the study further investigates the potential of AI applications in Social VR.

Proposed Solution

• Proposed Solution:

  • The authors propose addressing harassment in Social VR through collaboration among users, human moderators, and AI moderation systems (user-human-AI collaboration).
  • They suggest designing AI moderation systems that support real-time, large-scale harassment handling while considering socio-cultural contexts and incorporating community user feedback.

• Innovative Contributions:

  • The study introduces a new collaborative moderation model that positions AI as an indispensable team member rather than merely a tool.
  • It emphasizes building trust through transparent code and user-driven feedback mechanisms to mitigate potential power imbalances associated with AI in Social VR.

• Implementation Steps and Techniques:

  • Integrating the roles of users, human moderators, and AI through a multi-layered decision-making process, where each role handles distinct tasks:
    • Users are responsible for flagging potential harassment behaviors.
    • AI is used for large-scale monitoring and providing consistent preliminary judgments.
    • Human moderators focus on handling complex cases and making disciplinary decisions.
  • Promoting code transparency and AI customization (e.g., appearance and behavior) to enhance user control and trust.

Research Outcomes

• Specific Findings:

  • The study highlights the specific advantages of AI moderation, such as consistency, fairness, and scalability in addressing harassment.
  • It identifies three major limitations of AI moderation: lack of interpretability (e.g., understanding cultural contexts), technical constraints (e.g., detecting real-time voice harassment), and the potential to create new power imbalances.

• Advantages Compared to Existing Solutions:

  • Compared to traditional manual or community-driven moderation methods, AI moderation provides broader, real-time coverage of harassment behaviors in Social VR while reducing the emotional and labor burden on human moderators.
  • By incorporating user input and personalized design, the approach significantly enhances the acceptance and socio-cultural sensitivity of AI moderation.

• Experimental or Evaluation Results:

  • Interview data revealed user attitudes toward AI moderation, including trust, fairness, and perceived benefits, underscoring the importance of designing human-centered interactions for AI moderation.
  • Most users supported integrating AI into the collaborative system but emphasized the need to address fairness and transparency issues.

• Limitations and Future Directions:

  • Limitations:
    • The participant pool was primarily based in the United States, potentially leading to a narrow cultural perspective.
    • Most respondents lacked technical backgrounds, limiting in-depth exploration of technical improvements for AI moderation systems.
  • Future Directions:
    • Expand the survey sample to a global scale, including Social VR users from diverse cultures and backgrounds.
    • Further develop surveys and technical improvement studies to explore the creation of efficient AI systems capable of detecting synchronous harassment (e.g., real-time voice).

This research provides clear technical and design directions for creating safe and inclusive Social VR spaces while addressing theoretical and empirical gaps in HCI literature regarding AI moderation in emerging online environments.