How Do HCI Researchers Study Cognitive Biases? A Scoping Review

Research Background and Issues

• Issues and Challenges:
  The paper identifies cognitive biases as systematic deviations in decision-making and information processing, which can lead to interaction problems between users and computer systems. For instance, biases can exacerbate misunderstandings, errors in judgment, and make users more susceptible to algorithmic suggestions or manipulation.
  Although the HCI (Human-Computer Interaction) community has gradually begun studying cognitive biases, the research remains scattered across different academic fields and application scenarios, lacking comprehensive reviews and systematic understanding.

• Significance:
  Cognitive biases not only affect user-computer interaction behaviors but may also lead to real-world behavioral risks, including the spread of misleading information, unfair decision-making, and compromised data annotation quality. Meanwhile, designing systems that mitigate or leverage cognitive biases holds significant value. For example, such systems can improve user decision-making and facilitate behavior change.

• Research Motivation:
  This paper aims to provide a comprehensive review of the application of cognitive biases in HCI research to develop a systematic understanding. The objectives include mapping existing research, exploring applications across different domains, analyzing methods researchers use to study biases, and identifying gaps and potential opportunities in the field.

Solution

• Methodology and Framework:
  Using a scoping review approach, the authors systematically analyzed 127 articles published between January 2010 and May 2024, focusing on how HCI research quantifies, mitigates, leverages, and observes the effects of cognitive biases.

• Innovations and Key Points:

  1. Identified five primary approaches to studying biases in HCI research: studying effects, mitigating biases, observing biases, leveraging biases, and quantifying biases.
  2. Covered eight major HCI application domains: information interaction and recommendation systems, human-AI interaction, visualization, usability, behavior change, computer-supported cooperative work and social computing, robotics in HCI, and gaming.
  3. Proposed a framework diagram (Figure 1) that outlines the research dynamics from tool development, bias understanding, and design feedback to real-world user behavior.

• Implementation Steps:

  1. Data Collection: Retrieved papers containing terms related to "cognitive bias" from top HCI conferences (e.g., CHI, CSCW) and databases.
  2. Screening and Analysis: Applied the PRISMA method to filter out irrelevant or low-reliability case studies.
  3. Categorization and Coding: Extracted and openly coded core content, including definitions of cognitive bias terms, research focus, and application contexts.

Research Outcomes

• Specific Findings:

  • Summarized 92 core terms and definitions related to cognitive biases, with "confirmation bias" being the most studied.
  • Defined five primary research modes for cognitive biases and discussed the concentration and distribution of studies across eight application scenarios.
  • Highlighted the dual-edged nature of cognitive biases: they can be leveraged to induce positive behaviors (e.g., healthy choices or critical thinking) but can also be maliciously exploited to manipulate user behavior (e.g., dark pattern design).
  • Identified shortcomings in existing research, including inconsistent terminology usage, lack of theoretical frameworks, and limited studies on the interaction effects of multiple cognitive biases.

• Advantages Compared to Existing Technologies:
  The paper emphasizes its multidisciplinary perspective and comprehensive coverage of the HCI field. Its approach not only focuses on the negative effects of biases but also explores how design applications can shape positive bias utilization (e.g., tools for behavior change, educational modules).

• Experimental or Evaluation Results:

  • Established the significant impact of biases in domains such as information interaction and human-AI interaction.
  • For example, in "information interaction and recommendation systems," most of the 30 reviewed articles discussed bias issues in user information selection (e.g., confirmation bias).
  • Regarding bias mitigation strategies, researchers designed cognitive support tools that provide system feedback to help users reflect on biases.

• Limitations and Future Directions:

  • Limitations:
    • Inconsistent use of academic terminology posed challenges for the review, suggesting the need for standardized terminology.
    • Current research is predominantly based on laboratory settings, lacking real-world scenario data.
    • Limited comprehensive studies on the coexistence and interaction of multiple biases.
  • Future Directions:
    • Develop more universal tools for bias quantification.
    • Actively evaluate and improve the effectiveness and ethical implications of "nudging technologies" based on cognitive biases.
    • Expand research into underexplored application domains (e.g., gaming and human-computer collaboration).
    • Conduct longitudinal studies and contextual interaction experiments to observe the long-term and dynamic effects of cognitive biases.

In summary, this paper reviews the research on cognitive biases in the HCI field, revealing the current state, challenges, and opportunities. It calls for academic standardization and multidisciplinary collaboration to clarify bias definitions and advance more transparent technology design.