Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email Writing

Title of the Paper

Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email Writing

Paper Information

• Subject Area: Artificial Intelligence and Human-Computer Interaction, Trust, Communication Methods
• Keywords: Artificial Intelligence-Mediated Communication (AI-MC), Computer-Mediated Communication (CMC), Artificial Intelligence, Trust, Interpersonal Communication, Email Writing, High Interpersonal Emphasis, Topic Familiarity, User Experience

Research Background and Problem

• Problem and Challenges:
  With large language models capable of drafting or editing messages for users, the trustworthiness of AI-generated communication content may be affected. However, most recipients may not realize that these messages are AI-generated. Research and empirical evidence on how people perceive such AI-generated messages and their impact on communication trust remain scarce. This study focuses on the impact of AI on recipients' trust when drafting emails and explores the role of interpersonal emphasis in email content.

• Significance:
  As language model technology approaches human-level writing capabilities, AI-mediated communication is becoming increasingly prevalent. This not only affects language production and writing habits but may also alter trust and evaluation standards between individuals. Understanding the impact of AI writing tools on trust is crucial for guiding technology design and policy-making.

• Research Motivation and Related Work:
  The study addresses the trust issues faced by current AI systems in the language generation process, such as trust variations in transactional emails and highly interpersonal emails. Previous research has shown that AI-generated short messages or algorithmic recommendations may affect the sender's trustworthiness, but studies focusing on long-form email writing and different content contexts are unprecedented.

Solution

• Proposed Methods and Solutions:
  The study evaluates participants' trust in AI-generated emails through a combination of online surveys and in-depth interviews. A "Wizard-of-Oz" experimental design was employed, where all displayed emails were actually written by humans, but participants were told that the emails were written by humans, co-written by humans and AI (shared agency), or entirely AI-generated.

• Innovations:

  • Proposed a trust variation analysis framework based on perceptions of AI writing.
  • Used controlled experiments to exclude text quality as an evaluation factor, focusing on participants' qualitative and quantitative perceptions of AI involvement.
  • Explored the role of "interpersonal emphasis" in content, a variable that significantly influenced participants' trust.

• Implementation Steps and Techniques:

  1. Survey Design: Provided 12 emails categorized by low, medium, and high levels of interpersonal emphasis.
  2. Experimental Variables: Divided into AI conditions (human-written, AI-assisted, human-AI co-written) and levels of interpersonal emphasis (low, medium, high).
  3. In-depth Interviews: Collected participants' specific descriptions of trust and insights into appropriate scenarios for AI writing tools.
  4. Data Analysis: Conducted linear regression and variance analysis to validate hypotheses.

Research Findings

• Specific Findings:

  1. The more emails associated with AI, the lower the trust level. Fully human-written emails were the most trusted, while fully AI-written emails had the weakest trust.
  2. Nearly all participants believed that using AI in highly interpersonal scenarios (e.g., consoling someone for the loss of a pet) was inappropriate, yet quantitative data showed that highly interpersonal emails had the highest trust levels.
  3. Participants primarily evaluated emails based on content details and language style, often disregarding indications of AI involvement in writing.
  4. Topic familiarity had a significant positive impact on trust, indicating that participants familiar with the subject were more likely to trust messages written by either AI or humans.

• Comparison with Existing Solutions:
  Unlike previous studies on algorithm-recommended short replies, this study extends to long-form writing scenarios and uncovers new correlations between interpersonal emphasis and trust.

• Experimental or Evaluation Results:
  The experiment showed that as interpersonal emphasis increased, email trust ratings rose; however, trust in highly AI-mediated conditions remained lower than in human-written emails. Quantitative results confirmed the correlation and influencing factors of subjective trust ratings.

• Limitations and Future Directions:

  • Limited to email scenarios; future research could expand to instant messaging or cross-modal communication.
  • The experimental design was based on theoretically high-quality AI-generated texts; further research is recommended to study the effects of AI-generated writing in real-world contexts.
  • Since social background and culture may influence attitudes toward AI, expanding the research sample to diverse geographic and social environments is suggested.