Understanding User Reliance on AI in Assisted Decision-Making

Video

DOI

Proper calibration of human reliance on AI in assisted decision-making is fundamental to achieving complementary performance in human-AI collaboration. Most previous works focused on assessing user reliance, and more broadly trust, retrospectively, through user perceptions and task-based measures. In this work, we explore the relationship between eye gaze and reliance under varying task difficulty and AI performance levels in a spatial reasoning task. Our results show a strong positive correlation between total percent gaze duration on the AI suggestion and user AI task agreement, as well as user perceived reliance. Moreover, user agency is preserved particularly when the task is easy and when AI performance is low or inconsistent. Our results also reveal nuanced differences between reliance and trust. We discuss the potential of using eye gaze to gauge human reliance on AI in real-time, enabling adaptive AI assistance for optimal human-AI team performance.