For What It's Worth: Humans Overwrite Their Economic Self-Interest to Avoid Bargaining With AI Systems

Title of the Paper

For What It’s Worth: Humans Overwrite Their Economic Self-interest to Avoid Bargaining With AI Systems

Bibliographic Information

• Domain: Interaction between AI decision systems and human behavior
• Keywords: AI systems, online experiments, human-computer interaction, decision support systems, market interaction, ultimatum game
• Conference: CHI Conference on Human Factors in Computing Systems, April 29-May 5, 2022, New Orleans, LA, USA

Research Background and Problem

• Key Issue: This paper focuses on the potential impact of AI systems on human behavioral patterns when they replace or augment human decision-making, particularly in competitive environments where AI systems partially or fully substitute human participants. It examines how individuals choose and adopt behavioral strategies in such scenarios.
• Significance of the Study: As AI technology becomes widely applied in social and economic activities, understanding how these systems influence human decision-making is increasingly critical. This impacts not only direct users but also other stakeholders indirectly affected by AI-driven decisions, such as pricing algorithms in consumer markets, credit scoring systems, and workforce allocation in organizational contexts.
• Motivation and Related Work:
  • Many studies suggest that interactions with AI or algorithms may enhance market efficiency by reducing emotional involvement and the weight of social issues. However, they may also weaken cooperation and fairness, potentially harming economic and social outcomes.
  • Few studies have explored the comprehensive impact of algorithms on human beliefs, needs, and behaviors, particularly how economic and social motivations interact and evolve in environments involving artificial agents.

Solution

• Research Methodology: The authors designed an experiment based on the ultimatum bargaining game, analyzing the decision-making processes of 480 participants when interacting with three types of proposers (fully human, AI-assisted proposers, and fully AI agents) to uncover key behavioral patterns.
• Experimental Steps:
  1. Role Assignment: Participants were randomly assigned the roles of "proposer" or "responder" and completed corresponding tasks.
  2. Proposal Decisions: Proposers determined fund allocations, while responders decided whether to accept, reject, or predict proposer behavior in advance.
  3. Belief Incentive Measurement (BSR Method): Participants' beliefs about the allocation behavior of different proposers were measured to verify alignment with economic rationality.
  4. Self-selection Behavior: Participants were allowed to freely choose which of the three types of proposers to interact with, and their self-selection strategies and subsequent economic outcomes were studied.
• Technological and Methodological Innovations:
  • Combined belief incentive measurement (Belief Scoring Rule, BSR) with game-theoretic experimental methods to quantitatively analyze human economic and social preferences toward AI proposers.
  • Provided the first evidence that humans prioritize social factors over economic self-interest when interacting with AI agents, even "rewriting" their economic motivations.

Research Findings

• Key Experimental Results:
  1. Avoidance of AI Interaction: Most responders preferred human proposers (49%) over AI-assisted proposers (38%) or fully AI agents (12%). Fully AI-driven scenarios were the least favored.
  2. Economic Self-interest Replaced by Social Preferences:
     • Although responders predicted higher economic benefits from AI agents, only a small proportion chose to interact with them.
     • Responders demanded higher rewards when interacting with AI agents, reducing the likelihood of successful interactions.
  3. AI Introduction Reduces Economic Efficiency and Market Potential:
     • Fully AI-driven scenarios had a low success rate, meeting only 61% of demands, whereas AI-assisted proposals achieved nearly 100% success, indicating that AI struggles to adapt to or replicate the dynamics of complex social environments.
• Advantages and Impact:
  • Provided extensive experimental data demonstrating that the degree of autonomy in AI systems significantly influences human behavioral preferences and market efficiency.
  • Aids AI system designers in understanding user preferences and trust mechanisms, particularly in improving incentive schemes in competitive economic environments.
• Limitations and Future Directions:
  1. Lacks in-depth analysis of the motivations behind avoiding AI interactions, with only rough speculation that such behavior stems from social cognition factors like distrust or perceived unfairness.
  2. The experiment used an abstract game model, which only partially reflects real-world complexities. Future research should extend to other task domains (e.g., recruitment, healthcare, social fairness decisions).
  3. Further investigation is needed into how AI system transparency, experience accumulation, and explainability influence long-term human-AI collaborative behavior.