CommSense: Facilitating Bias-Aware and Reflective Navigation of Online Comments for Rational Judgment

Paper Title

CommSense: Facilitating Bias-Aware and Reflective Navigation of Online Comments for Rational Judgment

Publication Info

• Topic area: Human-Computer Interaction (HCI) and cognitive bias mitigation in online comment navigation.
• Keywords: cognitive bias, reflective judgment, online comments, interface design, bias mitigation, user-generated content, sensemaking, decision-making, visualization, HCI.

Background and Problem

• Problem / challenge: Online comment presentation, often algorithmically determined, can introduce cognitive biases like anchoring and confirmation bias, distorting user judgment. Existing research focuses on mitigating individual biases but overlooks how comment presentation affects the evolution of user reasoning.
• Significance: Bias in online comment navigation can lead to poor decision-making in critical domains such as travel, health, and politics. Addressing this issue can improve the quality of user judgments and promote balanced evaluations.
• Motivation and related work: Prior studies have explored bias mitigation through content moderation, rule-based filtering, and interface-level interventions. However, these approaches often neglect the dynamic interaction between user traits and comment presentation, as well as the need for tools that support reflective and evidence-based reasoning.

Solution

• Proposed approach: CommSense, a lightweight plugin designed to enhance user engagement with online comments by providing visual overviews, reflective prompts, and synthesis support to mitigate biases.
• Novelty:
  1. Identification of a four-stage decision path (Initial Framing, Evidence Foraging, Belief Updating, Synthesis & Judgment) and associated issues (goal alienation, evidence-belief loops, insufficient synthesis support).
  2. Development of design requirements for bias-aware interfaces: pre-engagement framing, interactive organization, in-situ reflective prompts, and dynamic synthesis support.
  3. Implementation of CommSense, integrating features like a Topic Corpus Overview, Comment Navigation Panel, and Synthesis Board to scaffold reflective decision-making.
• Procedure and key techniques:
  1. Conducted Study I (N=18) to trace user decision-making paths under different comment presentation conditions (Positive-First, Negative-First, Interleaved).
  2. Conducted Study II (N=18) to co-design interface features addressing identified challenges.
  3. Developed CommSense based on distilled design requirements.
  4. Evaluated CommSense in Study III (N=24) against a baseline interface, measuring bias awareness, reflective judgment, and usability.

Results

• Concrete findings:
  • CommSense improved bias awareness and reflective thinking, with participants producing more comprehensive, evidence-based rationales.
  • Significant reductions in cognitive workload (e.g., mental demand: CommSense mean = 3.17 vs. baseline mean = 4.58, p = 0.008).
  • Higher ratings for functionalities like overview providing (mean = 6.25/7) and evidence collecting (mean = 6.33/7).
• Advantage over baselines:
  • CommSense users demonstrated broader comment coverage, better sentiment balance (positive-to-negative ratio: 1.44 vs. baseline 1.13, p = 0.0133), and higher-quality final judgments (e.g., reflective depth: CommSense mean = 6.17 vs. baseline mean = 5.33).
  • Enhanced usability (System Usability Scale functionality: CommSense mean = 6.17 vs. baseline mean = 4.50, p = 0.0014).
• Experiments / evaluation:
  • Study I: Explored decision-making paths in a hotel evaluation scenario with 18 participants.
  • Study II: Co-design workshop with the same participants to derive design requirements.
  • Study III: Between-subjects evaluation (N=24) comparing CommSense with a baseline interface using metrics like usability, cognitive workload, and judgment quality.
• Limitations and future work:
  • Participant pool primarily consisted of graduate students, limiting generalizability.
  • Controlled experimental design may not fully capture real-world complexities.
  • Future work includes testing with diverse populations, enhancing customization, and applying CommSense to other domains like health and politics.

Summary

This paper introduces CommSense, a plugin designed to mitigate cognitive biases and support reflective judgment in online comment navigation. Through three studies, the authors identified a four-stage decision path, derived design requirements, and implemented CommSense, which integrates features like visual overviews, reflective prompts, and synthesis tools. Evaluation results demonstrated significant improvements in bias awareness, cognitive workload reduction, and judgment quality compared to a baseline interface. Future research will explore broader applications and user customization to enhance the tool's adaptability across diverse contexts.