"I See You!": A Design Framework for Interface Cues about Agent Visual Perception from a Thematic Analysis of Video Games

Title of the Paper

“I See You!”: A Design Framework for Interface Cues about Agent Visual Perception from a Thematic Analysis of Videogames

Paper Information

• Domain: Human-Computer Interaction, Human-Agent Collaboration Design
• Keywords: Human-Robot Interaction, Videogames, Visual Cues, Qualitative Analysis Framework, Cognitive Modeling

Research Background and Problem

• Problem or Challenge:

  • With the increasing prevalence of AI agents (e.g., robots, virtual assistants), it is necessary to enhance collaboration between humans and agents.
  • Humans need to understand what agents perceive, such as whether they “see” specific targets.
  • There is a lack of systematic frameworks for designing “visual cues.”

• Significance:

  • Providing visual cues can help humans better understand the perceptual and behavioral capabilities of AI agents, thereby improving safety, privacy, and efficiency.
  • These cues are particularly important for deploying robots in social contexts, such as domestic environments or public spaces.

• Research Motivation and Related Work:

  • Videogames have successfully designed various interactions over the years, especially visual cues. Understanding these design strategies can advance the field of human-computer interaction.
  • Reminder mechanisms in videogames, such as user interfaces based on visual targets and perception, can serve as inspiration for interaction design with artificial agents.
  • Previous research has focused on robot behavior transparency and animation principles but has overlooked the systematic communication of visual perception capabilities.

Solution

• Proposed Solution:

  • Develop a framework based on thematic analysis of visual cues in popular videogames to support the design of interactive interfaces that make agent perception visible to humans.
  • Create a classification framework for visual and auditory cues that describes the types of perceptual information they can convey (e.g., field of view, visual targets).

• Innovation of the Solution:

  • Drawing inspiration from game design to extract design principles from widely used and successful applications.
  • Enhancing situational awareness in human-computer interaction, helping users form accurate mental models of agent capabilities.

• Implementation Steps and Techniques:

  • Utilize Braun and Clarke’s thematic analysis method to conduct iterative research on multiple videogames.
  • Extract 60 instances of visual cues and identify key attributes such as cue origin, visualization/auditory methods, and types of information.
  • Define cue characteristics at multiple levels, including the presentation of detection information, visual target behavior, and perceiver behavior.

Research Outcomes

• Specific Outcomes:

  • Provide a comprehensive framework for designing visual cues (including both visualization and auditory modes).
  • The framework lists 13 types of cues suitable for design and validates their feasibility through actual game data.
  • Highlight the importance of “non-binary detection” visual cues, which represent intermediate states where the agent has not fully detected the target.

• Advantages:

  • Offers a thorough review of existing design practices, showcasing how to design visual interaction cues through game examples.
  • Optimizes user perception of agent capabilities, enhancing interaction efficiency and accuracy.
  • Provides a basis for selecting input modes (auditory, visualization) and cue types for designing robots and virtual assistants.

• Experimental or Evaluation Results:

  • The framework identifies multiple practical applications: for example, search-and-rescue robots need to provide clear and trustworthy visual signals; robots in public spaces require privacy-preserving cues.
  • The effectiveness of visual cues was validated using game examples, particularly in action and stealth game designs.

• Limitations and Future Directions:

  • Limitations:
    • Videogames primarily focus on visual perception design, with insufficient research on auditory or other sensory cues.
    • The framework emphasizes designs for adversarial relationships, lacking mechanisms for collaborative cues.
    • Videogame data may not fully replicate real-world interaction experiences, potentially overlooking localized auditory cue effects.
  • Future Directions:
    • Expand the design of cues to non-visual perception domains, such as auditory and tactile.
    • Investigate how to combine different types of cues to enhance readability, emotional characteristics, and information richness.
    • Test the framework’s compatibility in real-world robotic scenarios to improve hardware and user experience.

Conclusion

This paper proposes a framework for human-computer interaction design by analyzing visual cue designs in videogames. Years of interaction exploration in game design provide rich practical experience for design techniques. Future research will delve into the combined design of these cues and apply the findings to real-world scenarios involving robots and artificial agents to enhance collaboration efficiency and transparency between humans and intelligent agents.