Collaborating with a Text-Based Chatbot: An Exploration of Real-World Collaboration Strategies Enacted during Human-Chatbot Interactions

Title of the Paper

Collaborating with Text-Based Chatbots: Exploring Collaboration Strategies in Real-World Human-Chatbot Interactions

Paper Information

• Domain: Human-Computer Interaction, Chatbots, Collaboration Strategies
• Keywords: Chatbots, Human-Computer Collaboration, Task-Oriented Chatbots, Cooperation Strategies, Real-World Data, User Experience, Language Adaptation, Behavioral Strategies

Research Background and Issues

• Identified Problems or Challenges:

  • Most chatbot research relies on artificial experimental setups, lacking observations of real-world interactions.
  • Existing studies primarily focus on user responses to communication breakdowns, neglecting broader collaboration strategies throughout the conversation.
  • How humans collaborate with chatbots to achieve their goals remains insufficiently understood.

• Significance:

  • Chatbots are widely used in task-oriented domains like customer service, where dialogue quality directly impacts user experience.
  • A deeper understanding of human-chatbot collaboration dynamics can improve chatbot design and efficiency.

• Research Motivation and Related Work:

  • Current studies emphasize repairing broken conversations rather than analyzing overall collaboration strategies.
  • Analyzing real user chat logs offers opportunities to explore natural collaboration dynamics.
  • Some research has begun examining how users leverage chatbots to overcome issues, but systematic analysis of collaborative behaviors is still lacking.

Proposed Solution

• Proposed Approach:

  • Employ qualitative analysis methods (Grounded Theory) to analyze 12,477 interactions between real users (customers of an Italian telecommunications company) and a task-oriented chatbot.

• Innovative Aspects:

  • Unlike most studies relying on experimental or self-reported pseudo-realistic data, this study provides collaboration strategy analysis based on real-world chat logs.
  • Identified two major collaboration dimensions (behavioral collaboration and conversational collaboration) and their dynamic evolution.
  • Systematically revealed collaborative and non-collaborative methods and their evolutionary patterns for the first time.

• Implementation Steps and Key Techniques:

  • Data cleaning and preprocessing, establishing confidentiality and ethical guidelines.
  • Coding and classification based on Grounded Theory, analyzing behavioral and grammatical variations.
  • Identifying collaboration types, strategies, and interaction dynamics.

Research Findings

Specific Results

• Two Major Collaboration Dimensions:

  • Behavioral Collaboration: Users adapt to the interaction environment by following chatbot requests or maintaining politeness.
  • Conversational Collaboration: Users adjust language structure and content to facilitate chatbot comprehension.

• Collaboration Strategies:

  • Behavioral Strategies:
    • Commitment: Users strictly follow chatbot instructions.
    • Politeness: Users exhibit patience and polite dialogue through their language.
    • Behavioral Non-Collaboration: Users refuse to follow instructions or display aggressive behavior.
  • Conversational Strategies:
    • Summarization: Users reduce information to focus on key content.
    • Simplification: Transform complex syntactic structures into simpler sentences.
    • Reconstruction: Replace words to enhance chatbot understanding.
    • Complexification: Add structure and details to sentences.
    • Correction: Fix grammatical or spelling errors.
    • Emphasis: Use uppercase letters to highlight keywords.
    • Repetition: Repeat sentences verbatim when the chatbot fails to understand.
    • Conversational Non-Collaboration: Users do not simplify complex language or refuse to rephrase.

• Collaboration Dynamics:

  • Evolution: Users progressively enhance collaboration, such as shifting from complex sentences to simplified language and trying different strategies.
  • Degradation: Users gradually reduce collaboration, showing fatigue or frustration, potentially abandoning the conversation.

Advantages Compared to Existing Solutions

• Provides insights into real-world interactions, enhancing the reliability of research findings through authentic records.
• Covers a wide range of collaboration types, not limited to communication breakdown repair strategies.
• Captures dynamic evolution and degradation of collaborative behaviors.

Experimental or Evaluation Results

• Users demonstrated diverse collaboration strategies across numerous interactions.
• Over two-thirds of users persistently attempted collaboration despite communication barriers.
• Significant behavioral differences among users, with some exhibiting high levels of patience and politeness.

Limitations and Future Directions

• Limitations:

  • Chatbot functionality is limited, requiring users to bear the primary collaboration responsibility.
  • Data is confined to the cultural context of Italy, potentially limiting generalizability.
  • Lack of direct user reflections on their behavior prevents verification of behavioral motivations.

• Future Directions:

  • Extend research to other cultural contexts and different task-oriented chatbots.
  • Combine user reflections with chat logs to explore consistency between behavior and perception.
  • Develop intelligent chatbots with dynamic collaboration capabilities to better engage in human-computer collaboration.

Design Implications

1. Create a Friendly and Polite Conversational Environment:

   • Enhance chatbot emotional expression and human-like design.
   • Encourage users to adopt similar interaction styles through proactive polite behavior.

2. Guide Users to Adopt Appropriate Collaboration Strategies:

   • Provide clear guidance and dynamically suggest strategies (e.g., simplification or complexification).

3. Enable Users to Perceive Collaboration Progress:

   • Highlight subtle progress in interactions to prevent repetitive responses that make users feel the conversation is stagnant.

Conclusion

This paper provides a comprehensive perspective on collaborative behaviors in human-chatbot interactions within task-oriented contexts. The study reveals various strategies users employ to adapt to chatbots through behavioral and conversational means, analyzing the dynamic features of collaboration evolution and degradation. These findings offer concrete suggestions for improving chatbot design, particularly in customer service applications.