Help me and I’ll help you: Speakers’ and listeners’ collaborative effort and the division of labour in human-agent collaborative communication

Paper Title

Help me and I’ll help you: Speakers’ and listeners’ collaborative effort and the division of labour in human-agent collaborative communication

Publication Info

• Topic area: Human-agent collaborative communication and conversational AI design.
• Keywords: Collaborative effort, division of labour, conversational AI, human-agent interaction, visual ambiguity, visual asymmetry, partner models, speaker effort, listener effort.

Background and Problem

• Problem / challenge: Limited understanding of how agents’ collaborative effort affects users’ reciprocal effort and perceptions of agent conversational capability in human-agent collaborative communication (HACC).
• Significance: Effective collaboration between humans and conversational AI is critical for tasks involving shared effort, particularly in visually grounded contexts.
• Motivation and related work: Previous studies have explored isolated aspects of human-agent interaction, such as visual ambiguity and asymmetry, or user adaptation to perceived agent capabilities. However, these studies lack a unified view of speaker and listener effort and fail to systematically investigate how varying levels of agent collaborative effort influence user behavior.

Solution

• Proposed approach: Conduct an online director-matcher task experiment to systematically manipulate agents’ collaborative effort (over-informative, minimally informative, under-informative) and assess its impact on user effort and perceptions.
• Novelty:
  1. First application of the division of labour principle to agents’ collaborative effort.
  2. Integration of partner modelling and division of labour accounts to examine reciprocal user effort in HACC.
  3. Systematic manipulation of visual ambiguity and visual asymmetry in experimental design.
  4. Recommendations for context-sensitive over-informative strategies in conversational AI design.
• Procedure and key techniques:
  • Participants alternated between speaker and listener roles in a director-matcher task involving 3x3 grids of objects.
  • Agents varied in informativeness (over-informative, minimally informative, under-informative).
  • Visual context conditions included visual ambiguity (presence/absence of competitors) and visual asymmetry (common/privileged ground).
  • Measures included PMQ scores for partner models, listener effort (response time, cursor hovering, selection errors), and speaker effort (number of words, inclusion of features).

Results

• Concrete findings:
  • Over-informative agents were perceived as more competent and human-like than under-informative agents.
  • Listener effort was reduced with over-informative agents, leading to faster target identification, fewer errors, and less competitor consideration.
  • Contrary to expectations, users exerted more speaker effort with over-informative agents, emulating their collaborative behavior.
• Advantage over baselines:
  • Over-informative agents consistently outperformed minimally and under-informative agents in reducing listener effort and maintaining positive partner models.
• Experiments / evaluation:
  • Sample size: 267 participants.
  • Visual context manipulations: 3 speaker conditions, 4 listener conditions.
  • Metrics: PMQ scores, response times, hovering likelihood, selection errors, word count, feature inclusion.
• Limitations and future work:
  • Lack of feedback on task success may limit ecological validity.
  • Simplified experimental paradigm may not fully capture real-world complexities.
  • Future studies should explore naturalistic contexts, non-visual ambiguity, and counterbalance experimental conditions.

Summary

This study investigates how agents’ collaborative effort influences user behavior and perceptions in human-agent collaborative communication. Over-informative agents were found to reduce listener effort and improve perceptions of competence and human-likeness, while minimally and under-informative agents increased user effort and led to more negative partner models. Contrary to HHCC division of labour principles, users exerted more speaker effort with over-informative agents, emulating their collaborative behavior. These findings suggest conversational AI systems should adopt context-sensitive over-informative strategies, particularly in visually grounded tasks, to optimize collaboration and user experience.