Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation

Document Title

Human-AI Collaboration via Conditional Delegation: A Case Study of Content Moderation

Document Information

• Subject Area: Applications of Human-Computer Interaction and Artificial Intelligence in Content Moderation
• Keywords: AI collaboration, conditional delegation, content moderation, user interface design, machine learning, text classification, XAI (explainable AI), distribution shift

Research Background and Problem

• Problem or Challenge:

  • Although AI performs exceptionally well on certain benchmark datasets, its performance significantly degrades when handling out-of-distribution data. This can lead to erroneous decisions, particularly in large-scale, low-risk tasks such as social media content moderation.
  • Current human-AI collaboration approaches often emphasize human involvement in every decision, making it difficult to scale for tasks requiring rapid and efficient large-scale processing, such as content moderation.

• Significance:

  • AI models are unreliable under distribution shifts. There is a need to explore how to combine AI with human knowledge and judgment to enhance automation while retaining human control.

• Motivation and Related Work:

  • The field of human-AI collaboration has explored how to use AI for high-risk decision-making, but there is limited research on the applicability of conditional delegation in large-scale, low-risk tasks.
  • Current rule-driven automation systems (e.g., Reddit's AutoModerator) offer high controllability but fail to fully leverage AI's potential.

Solution

• Proposed Solution: Conditional delegation, which establishes human-AI collaboration by defining the trustworthy regions where AI can be relied upon to perform tasks.

  • Humans define these "trustworthy regions" before model deployment, and only content falling within these regions is handled by AI post-deployment. The remaining content is either managed by humans or handled through other measures.

• Innovations:

  • Introduced a new AI collaboration model that eliminates the need for humans to review each task individually, allowing humans to predefine rules that determine the scope of AI automation.
  • Integrated local and global explainability to assist humans in understanding model behavior.
  • Designed an experimental methodology to quantitatively validate the effectiveness of conditional delegation and study the successes and challenges of human rule creation in different scenarios.

• Implementation Steps:

  1. Interface Development: Designed and implemented a test system to support users in observing AI behavior through keyword searches and generating rules.
  2. Data Preparation and AI Modeling:
     • Used Wikipedia Attack Comments data (WikiAttack) as in-distribution data.
     • Used Reddit hate speech data as out-of-distribution data.
     • Employed an explainable neural network model that provides rationale-based predictions and analysis.
  3. Experimental Design:
     • Participants were tasked with writing keyword rules to evaluate the reliability of model predictions through conditional delegation.
     • Compared four conditions: predictions only, predictions + local explanations, predictions + local + global explanations, and current manual rule-based methods.
  4. Evaluation Metrics: Assessed rule quality and user experience using metrics such as precision, coverage, reward scores, and subjective user feedback.

Research Findings

• Key Findings:

  • Conditional delegation significantly improved model precision on in-distribution data (WikiAttack), outperforming the model operating independently.
  • On out-of-distribution data (Reddit), conditional delegation improved model performance but did not fully surpass manual rules.
  • Providing local and global explanations enhanced rule creation efficiency but could also mislead users into selecting inefficient keywords due to cognitive biases.

• Advantages Over Existing Solutions:

  • Balances automation with human agency, enhancing the scalability of collaborative decision-making.
  • Supports dynamic rule adjustments to adapt to different task scenarios, addressing challenges posed by distribution shifts.

• Experimental or Evaluation Results:

  • In in-distribution scenarios, local explanations significantly reduced users' subjective cognitive burden, increasing their confidence in task completion.
  • Global explanations shortened the time required for users to create rules, improving overall efficiency.
  • User-created rules effectively delineated "trustworthy regions" and optimized model prediction accuracy.

• Limitations and Future Directions:

  • The experiment participants were Mechanical Turk crowdworkers, lacking the perspective of professional content moderators.
  • The experiment only required participants to create a small number of rules; future research could explore how to motivate users to create more comprehensive rule sets.
  • The impact of distribution shifts on users' rule selection requires further detailed analysis.
  • Future work should investigate dynamic rule updating strategies for long-term deployment and proactive monitoring and control of biases in conditional delegation systems.