"AI enhances our performance, I have no doubt this one will do the same": The Placebo effect is robust to negative descriptions of AI

Title of the Paper

"AI enhances our performance, I have no doubt this one will do the same": The Placebo Effect Is Robust to Negative Descriptions of AI

Paper Information

• Field: Human-Computer Interaction (HCI) and Artificial Intelligence (AI) User Research
• Keywords: Placebo Effect, Decision-Making Process, Performance Expectation, Artificial Intelligence, User Experience, HCI, Negative Descriptions, Cognitive Modeling, Data Analysis, Task Design

Research Background and Questions

• Background:

  • The placebo effect—manifested in medical and HCI research as users perceiving improved subjective and objective performance when interacting with a fictitious AI system.
  • Previous studies suggest that negative expectations may lead to a "Nocebo effect," reducing cognitive function and performance.

• Research Questions:

  • For a non-existent AI system, does negative language reduce users' subjective performance expectations and objective task performance?
  • Can negative descriptions disrupt the placebo effect?
  • How does the placebo effect specifically influence users' decision-making processes (e.g., reaction time, accumulation of correct information, and other cognitive parameters)?

• Motivation:

  • Current AI user experience evaluation methods may be significantly influenced by user expectations (e.g., the placebo effect), potentially distorting assessments of AI systems.
  • By employing cognitive modeling and quantitative evaluation, this study aims to determine whether negative descriptions can eliminate the placebo effect and explore specific changes in the decision-making process.

Approach

• Research Methods:

  1. Experimental Design:
     • 28 participants were divided into "positive description" and "negative description" groups to complete a letter recognition task.
     • The task was conducted under two conditions: AI system "active" and "inactive" (no actual AI was present).
     • Participants' decision-making behavior, cognitive load, physiological signals, and subjective evaluations were monitored.
  2. Key Techniques and Tools:
     • The "Drift Diffusion Model (DDM)" was used to analyze decision-making processes.
     • Bayesian statistical methods and physiological signal recording devices (e.g., Electrodermal Activity, EDA) were employed for data collection and analysis.

• Innovations:

  • This study is the first to verify the robustness of the AI placebo effect by introducing negative language descriptions.
  • Cognitive modeling (e.g., DDM) was used to quantitatively test the impact of AI on intrinsic processes such as information accumulation speed (drift rate) and decision boundaries (boundary separation).

Research Findings

• Key Findings:

  1. The placebo effect is significantly robust to negative descriptions: even when participants were informed that AI might reduce task performance, they still expected AI to "help" improve their performance.
  2. Analysis using the Drift Diffusion Model revealed:
     • Drift Rate: Participants' information processing speed increased under the active AI condition.
     • Boundary Separation: Participants made more conservative decisions (requiring more information accumulation) when the system was "active."
     • Non-Decision Time: Decreased, particularly under the negative description condition.

• Comparison with Existing Research:

  • The study highlights that, contrary to previous conclusions about the placebo effect being influenced by descriptions, AI performance bias demonstrates "resistance" to negative descriptions.
  • Furthermore, negative language descriptions failed to induce a "Nocebo effect."

• Limitations:

  1. Emotional interference factors (e.g., positive emotions in the experimental environment suppressing negative expectations) may have influenced the results.
  2. Physiological load indicators (e.g., EDA signals) did not show significant differences, suggesting the need for more detailed task designs.
  3. The generalizability of the findings to broader tasks or cultural contexts requires further validation.

• Future Directions:

  1. Explore more complex AI contexts and real system operations, focusing not only on human-AI collaboration but also on competitive tasks.
  2. Develop strategies to control the placebo effect in practical applications and design.
  3. Enhance the efficacy of negative description interventions and investigate the role of emotions, cultural cognition, and other factors in shaping user expectations.