Bubbleu: Exploring Augmented Reality Game Design with Uncertain AI-based Interaction

Title of the Paper

Bubbleu: Exploring Augmented Reality Game Design with Uncertain AI-based Interaction

Paper Information

• Topic Area: Augmented reality game design based on uncertain artificial intelligence
• Keywords: augmented reality, computer vision, interaction design, human-computer interaction, uncertain AI

Research Background and Problem

• Identified Problems or Challenges:

  • Emerging augmented reality (AR) games leverage object detection technologies to enable realistic virtual interactions. However, the statistical nature of deep neural networks (DNNs) inevitably leads to recognition errors.
  • Object detection errors negatively impact the gaming experience, such as inaccurate object recognition or incorrect interactions.
  • Current research primarily focuses on the impact of AI uncertainty on interaction, leaving the gaming domain largely unexplored.

• Why This Problem is Important:

  • AR game design must address the decline in gaming experience caused by technological errors.
  • Errors in object detection not only affect user control but also compromise immersion and enjoyment in games.
  • As AI solutions are rapidly applied to AR scene analysis, exploring this area can enhance future gaming user experiences.

• Research Motivation and Related Work:

  • Previous studies have addressed uncertainty design in AI-driven applications such as chatbots and recommendation systems, but research on games, particularly AR games, remains insufficient.
  • The gaming domain is unique: players' tolerance for interaction and technical errors may differ from other fields.
  • Proposing design guidelines and mechanisms to improve gaming experiences under AI errors is a critical topic.

Solution

• Proposed Basic Solution:

  • Developed an AR pet-raising game, Bubbleu, based on object detection, to test three design principles: ambiguity, transparency, and controllability.
  • Through experiments and design improvements, the study explores how to mitigate object detection errors and optimize the gaming experience.

• Innovations:

  • The first academic design specifically addressing object detection uncertainty in AR games.
  • Proposed three directions for game design improvements (ambiguity, transparency, controllability) to address the impact of different object detection errors.

• Implementation Steps and Key Technologies:

  • Requirement Phase: Investigated the impact of object detection technology and its errors on game design, creating a basic version, "Bubbleu-Baseline."
  • Improvement Phase: Designed game variants incorporating ambiguity (hiding details to reduce error perception), transparency (providing users with detection explanations), and controllability (allowing users to adjust detection parameters).
  • Experiment Phase: Conducted gameplay trials with 36 participants under free and controlled conditions, observing user acceptance and feedback on the improved designs.

Research Outcomes

• Specific Outcomes:

  • The effectiveness of different design principles in improving user experience varied:
    • Ambiguity design successfully reduced users' perception of errors but could cause confusion about player intent.
    • Transparency design revealed errors and helped users understand the system, enabling them to improve object detection operations.
    • Controllability design enhanced players' sense of control over the system and increased interaction frequency but also raised the error rate.
  • In experiments, ambiguity design improved interaction success rates (e.g., by 20%), while transparency design enhanced successful experiences during free gameplay.

• Advantages Over Existing Solutions:

  • Provides a concrete design framework and strategies to better handle AI errors in the gaming domain.
  • Ambiguity and transparency designs offer practical references for rapidly expanding object detection applications in casual games.

• Experimental or Evaluation Results:

  • Players preferred transparency and controllability designs (especially the "custom threshold" feature), though different players had varying needs for each design.
  • While ambiguity design concealed errors, some players felt that the ambiguous expressions affected the clarity of interactions.

• Limitations and Future Directions:

  • Limitations:
    • Small user sample size (36 participants) from a single region (South Korea).
    • Limited exploration of game types and AI models, focusing only on pet-raising games.
    • Players' subjective preferences for certain features require further validation with larger samples.
  • Future Directions:
    • Expand research to a broader user sample and other regions.
    • Explore design optimizations in various game types, such as action games or AR social platforms.
    • Improve the robustness of AI models and refine system designs for diverse object detection technologies.

Through this study, the authors provide valuable guidance on using AI object detection in emerging AR interactive game design and offer a practical design framework for designers.