Persuasion in Pixels and Prose: The Effects of Emotional Language and Visuals in Agent Conversations on Decision-Making

Research Background and Issues

• What problems or challenges did the authors identify?
  This study focuses on the impact of AI-supported conversational agents (CAs) on user decision-making, particularly how emotional language and visual elements influence user trust, emotional resonance, and charitable donation behavior. The key issue is that, despite the increasing complexity of AI systems and their demonstrated persuasive abilities, the intricate relationship between these systems and user behavior remains insufficiently understood.

• Why is this issue important?
  With the widespread application of CA driven by large language models (LLMs) in fields such as healthcare, education, and finance, their potential to influence user behavior is becoming increasingly prominent. However, these technologies also pose risks of misuse, such as spreading misinformation or large-scale manipulation. This underscores the urgency of studying how AI influences decision-making and how to design these systems ethically.

• Research Motivation and Related Work
  The authors further explore the independent and combined effects of linguistic expression (positive or negative) and visual cues (positive or negative images) on user behavior. Related studies have shown that AI systems possess significant persuasive power in personalized interactions, but their relationship with long-term behavioral changes in users remains unclear. This study integrates these directions, aiming to reveal how CA language styles and emotional imagery influence charitable donation decisions.

Solutions

• What methods or solutions did the authors propose?
  The authors designed an online experiment where participants interacted with a CA exhibiting a predetermined attitude (positive or negative) while being exposed to visual cues (positive images, negative images, or no images) in a controlled condition.

• What is innovative about this solution?
  This study simultaneously evaluates the independent and combined effects of linguistic and visual elements, offering a new perspective on the complexity of AI-driven persuasive strategies. Additionally, the use of computationally generated images and large language models enhances the experimental design, representing a cutting-edge combination of technologies.

• What are the implementation steps? What key technologies were used?

  1. Experimental Design: Six experimental conditions (CA attitude × visual cue).
  2. Emotional Image Generation: Positive and negative wildlife images were created using AI tools like Midjourney and validated for emotional impact.
  3. Operational Logic: Participants interacted with the CA and completed donation-related tasks.
  4. Data Collection and Analysis: Linear regression and mediation analysis were used to evaluate the effects of visual cues and CA attitudes on trust, emotional resonance, and donation behavior.

Research Findings

• What specific findings were obtained?

  1. CA Attitude: A positive attitude significantly increased participants' trust in the CA, perceptions of benevolence, competence, and emotional closeness, but did not significantly increase donation amounts.
  2. Visual Cues: The presence of images (whether positive or negative) actually suppressed donation amounts; participants were more willing to donate and exhibited greater situational empathy in the absence of images.
  3. Combined Effects: The combined effects of CA attitude and visual cues did not significantly influence donation behavior.

• What advantages does it have compared to existing solutions?
  This study delves into the interactive effects of visual and linguistic factors on user behavior, challenging traditional views that emotional elements (especially visual cues) significantly enhance donation behavior.

• What were the experimental or evaluation results?

  1. Donation amounts under visual cue conditions were significantly lower than in no-cue conditions.
  2. Emotional resonance (e.g., empathy and emotional relevance) significantly increased donation willingness and mediated the effects of visual cues.
  3. CA language style (optimistic/pessimistic) influenced participants' trust and perceived competence of the system but had limited direct impact on behavior.

• Limitations and Future Directions
  Limitations:

  • The experimental environment was relatively controlled, and real-world charitable donations may be influenced by more complex factors (e.g., existing charitable reputation).
  • Although the sample size met statistical requirements, small effects may have gone undetected.
  • The ecological validity of artificial virtual scenarios may be limited, requiring more naturalistic research designs.

  Future Directions:

  • Investigate the persuasive effects of CAs in real-world environments, including longer interaction times and multi-turn system designs.
  • Extend research to other modalities, such as voice- or video-driven CAs.
  • Study the stylistic and cultural adaptability of visual content, such as differences in the acceptance of visual and linguistic cues across cultural contexts.
  • Incorporate longitudinal studies to examine changes in user trust and behavioral patterns over time.

By deeply exploring the dual elements of language and visual cues, the authors reveal the potential complexities of AI's role in human decision-making, while proposing ethical directions and improvement suggestions for future technology design. These findings not only deepen our understanding of AI-human interaction but also advance the research frontier of responsible design in socio-technical systems.