UnWEIRDing Peer Review in Human-Computer Interaction

Paper Title

UnWEIRDing Peer Review in Human-Computer Interaction

Publication Info

• Topic area: Biases in peer review processes within Human-Computer Interaction (HCI), focusing on Global South (GS) scholarship.
• Keywords: Peer review, epistemic oppression, Global South, Human-Computer Interaction, WEIRD bias, scholarly evaluation, epistemic injustice, academic publishing, inclusivity, SIGCHI.

Background and Problem

• Problem / challenge: Peer review in HCI disproportionately disadvantages Global South (GS) scholarship through biases that confine it to development research, dismiss theoretical contributions, and impose Western-centric norms. There is little systemic inquiry into these biases or their impact.
• Significance: These biases marginalize GS scholars, limiting their contributions to global knowledge production and perpetuating inequities in academic publishing.
• Motivation and related work: Prior studies have identified biases such as prestige, nationality, and language bias in peer review. However, the systemic mechanisms that perpetuate these biases, particularly against GS scholarship, remain underexplored. This paper addresses these gaps by examining the peer review process through the lens of epistemic oppression.

Solution

• Proposed approach: The paper uses focus groups with HCI researchers to analyze how WEIRD (Western, Educated, Industrialized, Rich, Democratic) biases manifest in peer review and proposes strategies to create a more equitable review process.
• Novelty:
  1. Systematic documentation of WEIRD biases in HCI peer review, including dismissal of GS theoretical contributions and monolithic assumptions about GS contexts.
  2. Examination of reviewers’ experiences, including tokenization as "cultural experts" and the hidden curriculum of Western-centric writing norms.
  3. Application of Dotson’s epistemic oppression framework to peer review, proposing first-, second-, and third-order changes for systemic reform.
  4. Concrete recommendations for authors, reviewers, and HCI venues to address biases and foster inclusivity.
• Procedure and key techniques:
  • Conducted four focus groups with 16 HCI researchers experienced in GS scholarship.
  • Thematic analysis of focus group transcripts using Dotson’s epistemic oppression framework.
  • Proposed actionable strategies for authors, reviewers, and conferences based on participant insights.

Results

• Concrete findings:
  • 63% of participants reported receiving biased reviews targeting GS contexts.
  • Common biases included questioning the novelty of GS research, dismissing theoretical contributions, and imposing Western-centric standards of generalizability.
  • GS researchers faced additional burdens such as over-explaining cultural contexts and being tokenized as "cultural experts."
• Advantage over baselines:
  • Offers a systemic critique of peer review, moving beyond individual biases to address structural inequities.
  • Proposes multi-level changes (first-, second-, third-order) to counteract epistemic oppression in HCI.
• Experiments / evaluation:
  • Focus groups with 16 participants representing diverse GS contexts and career stages.
  • Participants had an average of 8 years of HCI research experience and included authors and reviewers for major HCI venues.
  • Analysis produced 438 unique codes grouped into 3 first-level themes and 13 second-level themes.
• Limitations and future work:
  • The study is limited to the perspectives of 16 participants, most of whom are affiliated with Global North institutions.
  • Future work could explore the perspectives of reviewers and authors from underrepresented GS regions and extend the analysis to other disciplines.

Summary

This paper examines how WEIRD biases in HCI peer review marginalize Global South (GS) scholarship, using focus groups with 16 researchers and Dotson’s epistemic oppression framework. Key findings reveal systemic biases that dismiss GS theoretical contributions, impose Western-centric norms, and tokenize GS reviewers as "cultural experts." The authors propose first-, second-, and third-order changes, including reviewer training, codifying inclusive review standards, and systemic reforms to reduce epistemic harm. These recommendations aim to foster a more equitable and inclusive peer review process in HCI, benefiting GS scholars and the broader research community.