"Pikachu would electrocute people who are misbehaving": Expert, Guardian and Child Perspectives on Automated Embodied Moderators for Safeguarding Children in Social Virtual Reality

Title of the Paper

"Pikachu would electrocute people who are misbehaving”: Expert, Guardian and Child Perspectives on Automated Embodied Moderators for Safeguarding Children in Social Virtual Reality

Paper Information

• Research Area: Children's online safety, social virtual reality environments, artificial intelligence, and human-computer interaction
• Keywords: Social virtual reality, metaverse, children's online safety, parents, guardians, design workshops, expert interviews, artificial intelligence, embodied moderator management

Research Background and Issues

• Problems or Challenges:

  • The immersive nature and avatar-based interactions of social virtual reality platforms raise potential harassment issues in spaces where children and adults coexist.
  • Current human or rule-based online safety tools (e.g., blocking, reporting) fail to ensure real-time responses and address complex environments effectively.
  • Existing technologies do not support remote supervision of children's interactions, and parents may lack awareness of their children's experiences on these platforms.
  • Human moderators are insufficient to handle all harassment incidents, with only 24% of such behaviors being actively monitored.

• Significance:

  • Children's experiences in social virtual reality (Social VR) can profoundly impact their psychological and personality development. Compared to social media, the immersive interactions in these spaces make children more vulnerable to the trauma caused by harassment.
  • Current tools have technical and implementation blind spots, posing challenges for children and their guardians while hindering improvements in platform safety.

• Research Motivation:

  • Propose automated embodied moderators (AEMs) that combine artificial intelligence and avatar technology to explore their feasibility in protecting children.
  • Investigate the design possibilities, intervention strategies, and practical needs of AEMs from the perspectives of children, parents, and online child safety experts.

Solution

• Methods and Solutions:

  • Introduce an avatar-based automated AI moderator (AEM) capable of intervening in harassment behaviors in social VR in a real-time, scalable, and approachable manner.
  • Combine insights from expert interviews and family-participatory design workshops to propose specific designs and functional requirements for the system.

• Innovations:

  • Integrate perspectives from multiple stakeholders (children, parents, experts), emphasizing personalized customization, age-appropriate responses, positive reinforcement, and psychological counseling. Practical suggestions for subsequent technological development are derived through participatory design.
  • Unlike traditional text or voice notification methods, AEMs focus on direct intervention in immersive VR environments through socially embodied avatars, enhancing children's trust and engagement.

• Implementation Steps and Key Technologies:

  1. Expert Interviews:
     • Conduct research with 16 experts in the field of online child safety to gather opinions on the use of automated embodied moderators.
  2. Family Design Workshops:
     • Engage 13 children and 8 parents in design workshops to collect ideas for avatar and moderator design, intervention preferences, and appearance suggestions.
  3. Design and Scenario Simulation:
     • Use case-based simulations (e.g., "Big Buddy" avatar moderator) to demonstrate potential intervention methods such as warnings, blocking, and parental notifications.
  4. Data Analysis and Theme Extraction:
     • Apply thematic analysis to extract core design principles from interviews and workshops, optimizing the research direction based on participant feedback.

Research Findings

• Specific Outcomes:

  • Proposed three core modules for AEMs: automated detection, contextual intervention, and user support.
  • Recommended designing psychological support, educational reprimands, warnings, and reconciliation mechanisms separately for "victims" and "harassers," advocating for a gradual intervention strategy.
  • Detailed design suggestions include age-group-based customization of appearance and behavior, allowing children to choose trusted characters (e.g., cartoon figures or pets).

• Advantages of Existing Solutions:

  • AEMs provide round-the-clock immediate responses, unlike the inconsistent availability of human moderators.
  • Compared to traditional technical standards, AEMs offer greater approachability, such as protecting children through non-verbal gestures like virtual hugs.
  • Focused on cross-group (children, parents, experts) concerns with high consensus, avoiding one-size-fits-all processes.

• Experimental and Evaluation Results:

  • The study found that AEMs significantly reduce harassment behaviors in social VR and can quickly intervene in severe incidents while providing follow-up remediation.
  • Children prefer friendly avatar designs, while parents emphasize enhanced system fairness and transparency.
  • Experts suggest that AEMs should balance behavioral guidance and emotional support to avoid alienating children or reinforcing surveillance fears (e.g., "panopticon effect").

• Limitations and Future Directions:

  • Limitations:
    • Data primarily sourced from the UK, with cultural and social differences potentially affecting the generalizability of conclusions.
    • The actual design and testing process requires broader, long-term experimental validation.
  • Future Directions:
    • Explore integration with more real-world scenarios, dynamic intervention mechanisms, and continuous learning capabilities.
    • Enhance human-AI collaboration mechanisms (e.g., semi-automated management) to address potential errors in pure AI models.
    • Conduct in-depth studies on the long-term psychological and behavioral impacts of AEM designs on children.