Effects of Communication Directionality and AI Agent Differences in Human-AI Interaction

Title of the Paper

Effects of Communication Directionality and AI Agent Differences in Human-AI Interaction

Paper Information

• Subject Area: Human-AI Interaction, Collaborative Games, Social Psychology
• Keywords: Artificial Intelligence, Human-AI Interaction, Cooperative Games, Social Perception, Communication Directionality

Research Background and Questions

Research Background

• With the widespread application of artificial intelligence (AI), the performance of AI in collaborative environments and users' perception of it have garnered significant attention.
• Social perception (e.g., intelligence, likability, and interactional trust) is crucial for the success of human-AI collaboration.
• In Cooperative Partially Observable Games (CPO), communication directionality (e.g., who leads the communication) and the perception of AI identity may influence users' experiences and evaluations of their collaborative partners.

Research Questions

The authors propose two main research questions:

1. How do communication directionality (user as information sender or receiver), AI type, and identity perception (AI or human) affect users' social perception of their collaborative partners in AI-driven cooperative games?
2. How do communication directionality, AI type, and identity perception influence collaborative outcomes in cooperative games?

Significance

• The current study fills a research gap by exploring how various AI behavior patterns and communication directionality impact users' social perceptions and collaborative performance.
• The authors aim to reveal how to optimize human-AI collaboration scenarios to enhance collaboration efficiency and user satisfaction through experiments.

Solution

Methods and Experimental Design

• Experimental Platform: An online cooperative word-guessing game, "Guess the Word."
  • Game Rules: One player (or AI) acts as the “giver” providing hints, while the other player guesses the target word.
  • Players must use clues and interact with each other to achieve the goal.
• AI Models:
  • Model A: Uses supervised learning techniques to generate hints based on an associative word database and semantic networks.
  • Model B: Employs reinforcement learning to improve performance through self-play.
  • Model C: Utilizes a data-driven approach, generating hints using a word association network.
• Experimental Variables:
  • User Role: Hint giver (“giver”) and guesser (“guesser”).
  • AI Type: Three AI models with different behavioral characteristics.
  • Identity Perception: Users were informed whether their partner was an AI or a human.
• Data Collection:
  • 199 participants completed 10 rounds of the game and filled out a social perception questionnaire evaluating their partner on dimensions such as intelligence, likability, and affinity.

Innovations

1. Multi-Perspective Study:
   • Covers AI role behavior (giver and guesser), different AI types, and boundary conditions of user perception.
2. Integration of Social Perception and Outcome Analysis:
   • Examines the relationship between social psychology variables (e.g., intelligence, likability) and actual game performance.

Research Findings

Specific Results

1. Differences in Social Perception:

   • User role (giver/guesser) and AI model type significantly influenced evaluations of AI intelligence, likability, and affinity.
   • In the “guesser” role, Model C was perceived as more efficient and likable (e.g., fewer rounds and higher likability scores).
   • In the "giver" scenario, users felt that “leading the interaction” provided them with greater control, resulting in higher intelligence ratings for the AI partner.

2. Impact of Identity Perception:

   • Compared to believing their partner was human, users exhibited slightly more negative social perceptions when they knew their partner was an AI. However, this effect varied depending on the AI model's responsiveness.
   • Model C, due to its behavior aligning closely with user expectations, reduced negative biases.

3. Differences in Collaborative Outcomes:

   • Collaboration efficiency (e.g., number of rounds used) when interacting with AI varied based on communication directionality and the model's adaptability.
   • When users believed their partner was human, they tended to repeat hints or provide complex clues, which AI struggled to utilize effectively, leading to decreased efficiency.

Comparison with Existing Solutions

• Advantages:
  • Provides a multidimensional perspective on how differences in AI behavior impact user perception.
  • Systematically explores optimization pathways for AI communication characteristics in partially observable cooperative scenarios.
• Disadvantages/Limitations:
  • The experimental scenario is limited to a word-guessing game, and the applicability of findings to real-world collaborative environments requires further validation.

Experimental Limitations and Future Directions

1. Real-World Applicability:
   • While the word-guessing game serves as a suitable testbed, its relatively simple task format limits the generalizability of findings to more complex real-world environments.
2. Future Directions:
   • Extend research to cross-domain fields such as education and healthcare collaboration.
   • Investigate dynamic changes in social perception rather than static conclusions.
   • Incorporate more individual difference factors, such as cultural background and cognitive styles.

Conclusion

This study reveals the profound impact of communication directionality and AI behavior on users' social perceptions and collaboration outcomes. It provides valuable insights for optimizing human-AI collaboration and designing AI systems.