The Design and Development of a Game to Study Backdoor Poisoning Attacks: The Backdoor Game

Document Title

The Design and Development of a Game to Study Backdoor Poisoning Attacks: The Backdoor Game

Document Information

• Subject Area: Human-Computer Interaction and AI Security
• Keywords: backdoor poisoning, activation clustering, AI security, gamification, crowdsourcing

Research Background and Problem

• Identified Issues and Challenges:

  • AI models rely heavily on large amounts of training data, which makes the data vulnerable to attacks, especially when using crowdsourced datasets.
  • Backdoor attacks represent a novel threat, where attackers maliciously modify a small portion of the training data (e.g., images with special objects) to potentially influence the model's classification capabilities.
  • Backdoor attacks are difficult to detect because the model performs normally on inputs without backdoor triggers but fails under specific trigger conditions.
  • There is currently a lack of platforms to study the specific mechanisms and effects of backdoor attacks.

• Significance:

  • Backdoor attacks pose a serious threat to the security of AI systems, particularly in fields like image recognition and autonomous vehicles that rely on deep learning.
  • Understanding and defending against these threats is crucial for improving model robustness.

• Research Motivation:

  • Given the novelty of backdoor attacks and the lack of research tools, the authors aim to create an interactive system to simulate and study backdoor attacks.
  • Designing a game serves as a means to encourage active participation in data collection and analysis, while also bridging collaboration with non-expert users.

Solution

• Proposed Method:

  • Design a game called “Backdoor Game” that allows users to interact with classifiers affected by backdoor attacks and upload images containing backdoor triggers.
  • Utilize the “Activation Clustering” technique to help users identify tampered data nodes through clustering methods.
  • Employ gamification to encourage users to explore backdoor instances and contribute images for research.

• Innovative Aspects:

  • This is the first attempt to connect AI security research with non-expert users through gamification, enabling large-scale collection of backdoor object data.
  • Provides a comprehensive platform to analyze and compare the effectiveness of various backdoor objects while drawing design inspiration from user behavior.

• Implementation Steps and Key Technologies:

  1. Game Challenge Design:
     • Synthesize a large amount of training data with backdoor triggers, creating classifiers (e.g., dog/cat classifiers) using the Open Images dataset.
     • Use Activation Clustering to cluster training data into “clean” and “potentially contaminated” images.
  2. Gamified Reward Mechanism:
     • Provide users with 10 initial “exploration” opportunities (peeks), allowing them to click on nodes to check whether images are contaminated; additional exploration opportunities are granted after submitting guesses.
     • Implement a “blur level” mechanism to restrict users’ clear access to training images, with the blur decreasing as correct identifications increase.
  3. User Interaction and Data Collection:
     • Players can submit images (e.g., misclassified photos with triggers) and text descriptions of their guesses.
     • The system analyzes submitted data to evaluate the misleading effectiveness of backdoor objects.

Research Outcomes

• Specific Findings:

  • Efficiency Comparison: Certain backdoor trigger objects (e.g., tennis balls) were found to be more effective than others (e.g., carrots, forks), as their image features were more prone to model misclassification.
  • User Behavior Observation: Users’ exploration behaviors and uploaded images revealed their guessing strategies and preferences regarding backdoor triggers. Customization in challenge design enhanced user engagement.
  • Diverse Data: The game collected image submissions from various scenarios, including original user contributions (e.g., household pets with backdoor objects).

• Advantages:

  • Compared to traditional methods, the Backdoor Game offers an innovative platform for public participation, enabling large-scale collection of diverse data to observe backdoor attack types.
  • Encourages non-expert involvement in AI security research while providing researchers with richer datasets for analyzing backdoor attacks.

• Experimental Results Evaluation:

  • Deployment results showed that out of 68 Mechanical Turk users, 66% completed the challenge, with some users misusing specific images to generate backdoor objects.
  • Among submitted data, 39% were correct results, and the diversity and creativity of the data demonstrated the game’s potential for data collection.

• Limitations:

  • The current system focuses on simplified binary classification tasks (cat/dog classifier), and further expansion is needed for more complex models.
  • Training models with synthetic backdoor data may not fully align with real-world scenarios.
  • Some users submitted unrelated images (“random photos”) to gain more exploration opportunities, which affected data quality.

• Future Research Directions:

  • Expand the scope to multiclass tasks and support datasets from different domains (e.g., medical imaging, autonomous driving).
  • Further optimize the game’s reward mechanism and user interface design to mitigate the issue of “random submissions.”
  • Test different backdoor objects (e.g., watermarks, single-pixel errors) on real-world datasets to provide more representative training sets.
  • Explore more comprehensive detection algorithms, including activation patterns in other layers of neural networks.
  • Extend the system to mobile platforms to better support users in submitting original data anytime, anywhere.

Conclusion

Through the design and development of the Backdoor Game, the authors present an innovative interactive tool for exploring backdoor attacks and their impact on deep learning models. This research opens new pathways in AI security and human-computer interaction while providing a framework for future exploration of backdoor attack studies and defense strategies.