Belief Updating and Delegation in Multi-Task Human–AI Interaction: Evidence from Controlled Simulations

Paper Title

Belief Updating and Delegation in Multi-Task Human–AI Interaction: Evidence from Controlled Simulations

Publication Info

• Topic area: Human–AI interaction in multi-task settings, focusing on belief updating and delegation.
• Keywords: belief updating, delegation, human–AI interaction, multi-task AI, Bayesian updating, trust in automation, conservatism bias, dispositional trust, calibration, reliance.

Background and Problem

• Problem / challenge: Existing research on human–AI interaction often focuses on single-task scenarios, leaving a gap in understanding how users update beliefs and calibrate reliance when interacting with multipurpose AI systems across tasks with varying accuracy.
• Significance: As large language models (LLMs) are increasingly used for diverse tasks within a single interface, understanding belief dynamics and delegation decisions is critical for designing reliable and user-friendly AI systems.
• Motivation and related work: Prior work has explored trust calibration, belief updating, and reliance in single-task settings, but has not addressed how users transfer beliefs across tasks or how dispositional traits influence reliance. This study aims to fill this gap by examining belief spillovers, bounded rationality, and the interplay of trust and confidence in multi-task human–AI interaction.

Solution

• Proposed approach: A controlled experiment simulating interactions with a multipurpose AI system across three tasks (grammar checking, travel planning, visual question answering) with fixed accuracy levels.
• Novelty:
  1. Systematic evidence of belief spillovers across tasks, showing path-dependent expectations.
  2. Quantification of bounded rationality in belief updating, revealing conservatism bias.
  3. Analysis of how subjective beliefs and self-confidence jointly shape delegation decisions.
  4. Identification of dispositional trust and AI literacy as predictors of initial priors.
• Procedure and key techniques:
  • Participants (N = 240) completed three tasks in randomized order with AI assistance at fixed accuracy levels (30%, 60%, 90%).
  • Belief updating was benchmarked against Bayesian norms using a Beta–Binomial model.
  • Delegation decisions were operationalized as binary choices (self vs. AI).
  • Pre-survey measures captured dispositional trust, AI literacy, and cognitive traits.
  • Trial-by-trial data on beliefs, confidence, and delegation were collected and analyzed.

Results

• Concrete findings:
  1. Beliefs did not reset across tasks; a 10-point increase in posterior beliefs from one task predicted a 3–4 point higher prior in the next task.
  2. Within tasks, belief updates followed the Bayesian direction but were conservative, proceeding at ~50% of the normative rate.
  3. Delegation was primarily driven by subjective beliefs about AI accuracy, with higher beliefs increasing reliance.
  4. Self-confidence independently reduced delegation when beliefs were held constant.
  5. Dispositional trust and AI literacy predicted higher initial priors about AI accuracy.
• Advantage over baselines: The study provides the first systematic evidence of belief spillovers and bounded rationality in multi-task AI settings, offering insights into user behavior that deviate from rational Bayesian benchmarks.
• Experiments / evaluation:
  • Tasks: Grammar error detection (30% accuracy), travel planning (60%), visual question answering (90%).
  • Metrics: Belief trajectories, delegation rates, Bayesian updating coefficients, and dispositional trust measures.
  • Robustness checks: Mixed-effects models, OLS regressions, and alternative counterfactual policies for Bayesian updates.
• Limitations and future work:
  • Tasks were limited to three domains and used pre-scripted AI outputs, which may not fully capture real-world LLM variability.
  • Immediate feedback on correctness was provided, which may not generalize to all AI applications.
  • Future work should explore dynamic accuracy, iterative interactions, and additional domains such as coding or creative tasks.

Summary

This study investigates belief updating and delegation in multi-task human–AI interaction, revealing that users exhibit belief spillovers across tasks, conservative belief updating, and reliance decisions driven by subjective beliefs and confidence. Dispositional trust and AI literacy also shape initial expectations. These findings highlight the need for AI systems to provide task-specific performance cues and calibrated feedback to mitigate belief inertia and foster appropriate reliance. The results offer actionable insights for designing multipurpose AI systems that support well-calibrated trust and safe delegation.