Do Expressions Change Decisions? Exploring the Impact of AI's Explanation Tone on Decision-Making

Research Background and Issues

• Problem Identification: The authors observed that while interpretative information provided by AI-driven decision support systems can assist users in evaluating recommendations, there is limited research on how the tone of these explanations (e.g., formal or humorous) affects human decision-making.
• Importance of the Problem: The way explanations are expressed may influence users' trust and decision-making, which is particularly critical in high-risk domains such as judicial or medical contexts. This could lead to users either over-relying on the system or neglecting important recommendations, thereby impacting the system's reliability.
• Research Motivation and Related Work:
  1. Previous studies have shown that the content of AI-generated explanations significantly affects users' trust and task performance. However, research on "modes of expression" has primarily focused on users' perceptions of AI rather than its actual impact on decision-making.
  2. Users' sensitivity to tone and expression may vary depending on the role of AI and user attributes, warranting further exploration of these complex interactions.

Solution

• Proposed Approach:
  1. Investigate the impact of different tones (e.g., formal, humorous) on user decision-making.
  2. Examine the relationship between tone and user decisions across different AI roles (assistant, second opinion provider, expert).
  3. Analyze how user characteristics (e.g., age and personality traits) moderate the effects of tone on decision-making.
• Innovative Contributions:
  1. Introduced the concept of "tone-based explanation moderation," emphasizing the potential influence of tone and expression on user behavior.
  2. Designed three experimental scenarios (movie recommendation, opinion formation, criminal recidivism risk prediction) representing different AI roles.
  3. Leveraged large language models to generate diverse explanation datasets.
• Implementation Steps:
  1. Utilize large language models to generate experimental data, including movie plots, crime analysis, and explanations in various tones.
  2. Conduct phased testing of user responses to explanations with different tones, including a background introduction phase (no explanation), a pre-intervention phase (neutral tone explanations), and an intervention phase (specified tone explanations).
  3. Assess changes in user decisions and their correlation with user characteristics through questionnaires and psychological measurements.

Research Outcomes

• Specific Findings:
  1. In the opinion formation scenario, the tone of explanations significantly influenced user decisions, regardless of user attributes.
  2. In other scenarios (movie recommendation and criminal risk prediction), the strength of tone's impact varied based on user attributes. For example, older users were more inclined to accept formal tones, while humorous tones were more effective for younger users.
  3. The study revealed that users expect tone adjustments to enhance information clarity, empathy, and diverse perspectives, but also expressed concerns about potential manipulation and inconsistencies in tone leading to bias.
• Advantages of Existing Solutions: Unlike traditional content-based explanations, this study systematically evaluates the impact of "tone of expression" on decision-making across multiple scenarios and explores the complex interactions between user characteristics and tone.
• Experimental or Evaluation Results:
  1. In the opinion formation scenario, humorous tones were perceived as lacking seriousness, which affected the credibility of the information.
  2. In the criminal risk prediction scenario, users adopted different information processing strategies (sequential or simultaneous referencing), and tone had varying degrees of influence on task decisions.
  3. In the movie recommendation scenario, the priority of explanation content was significantly higher than tone.
• Limitations and Future Directions:
  1. The types of tones used (e.g., humorous, formal) may not fully encompass the diversity of expressions.
  2. Sample size imbalances in certain scenarios may have affected statistical power.
  3. Participants' preconceived expectations of AI may introduce bias, requiring more systematic control of these influences.
  4. Future research should explore more complex styles of expression and the interaction between task characteristics and tone.

Through the above analysis, this study provides valuable insights for designing effective and user-friendly interpretative AI systems, particularly in adjusting the tone of expression to meet diverse user needs and task scenarios.