Why, when, and from whom: considerations for collecting and reporting race and ethnicity data in HCI

Document Title

Why, When, and From Whom: Considerations for Collecting and Reporting Race and Ethnicity Data in HCI

Document Information

• Subject Area: Collection and reporting of race and ethnicity data for research participants in the field of Human-Computer Interaction (HCI) and its implications
• Keywords: Race, Ethnicity, Systematic Literature Review, HCI Research, Surveys

Research Background and Issues

• Issues and Challenges: Studies reveal that reporting race and ethnicity information for research participants in the HCI field is extremely rare (less than 3%). Existing guidelines on "when, why, and how" to collect such data remain underdeveloped, and researchers lack consensus on these issues.
• Significance: Diverse participants are critical for developing safe, inclusive, and equitable technologies. The lack of race and ethnicity data may lead to biased understandings of technology use and impact, further exacerbating systemic inequalities in technological systems.
• Research Motivation and Related Work:
  • Research on race and ethnicity in other fields (e.g., medicine and psychology) has been shown to provide important insights into health and social equity.
  • Studies on gender and class have received more attention, but race as a critical dimension remains underexplored.
  • Tools such as Critical Race Theory have been used to analyze racial issues and biases in HCI, but further exploration is needed to understand how participant racial composition affects technology design and use.

Proposed Solution

• Proposed Solution:
  • Conduct a systematic literature review of CHI conference papers from 2016 to 2021, combined with a survey of relevant authors, to explore the current state of reporting race and ethnicity data in HCI research and the reasons behind it.
  • Provide multidisciplinary recommendations and considerations to help HCI researchers make informed decisions about collecting race and ethnicity data.
• Innovations:
  • Propose detailed considerations, including why, when, and from whom to collect race and ethnicity data.
  • Combine literature review and survey data to develop a multidimensional analytical framework to help researchers mitigate potential risks associated with data collection and usage.
• Implementation Steps and Techniques:
  • Develop a multi-layered literature screening method combining keyword filtering, manual analysis, and surveys to identify HCI studies reporting race and ethnicity data.
  • Analyze trends in racial composition and specific methods used in the studies (e.g., survey design).
  • Summarize motivations and practices for collecting race and ethnicity information through open-ended surveys with relevant researchers.

Research Outcomes

• Specific Outcomes:
  1. Analysis Results:
     • From 2016 to 2021, less than 3% of CHI conference papers reported race and ethnicity information.
     • Among all participants, 64.1% were non-Hispanic White, 10.3% were Black, 8.9% were Hispanic, 6.7% were Asian, and 4.6% belonged to other racial groups.
     • The racial composition of study participants differed slightly from U.S. Census statistics or FDA clinical trial results.
  2. Survey Findings:
     • Researchers' primary motivations for collecting race and ethnicity data included validating external validity, exploring race-specific phenomena, and complying with external requirements (e.g., U.S. government regulations).
     • Data collection methods varied, including single-choice, two-question, and open-ended responses in survey designs.
  3. Multidisciplinary Perspectives and Practical Recommendations:
     • Clear design guidelines were provided, such as allowing multiple-choice answers or open-ended options in surveys to help participants self-identify.
     • Researchers were encouraged to use mixed-race labels cautiously in quantitative analyses and avoid defaulting to a single cultural group (e.g., White) as the reference group.
• Advantages:
  • Compared to previous fragmented discussions, this study systematically integrates the current state and methods of reporting race information in HCI, providing clear references for future research.
  • The recommendations emphasize conducting research in a precise and diversity-respecting manner, achieving a better balance between theory and practice.
• Experimental or Evaluation Results:
  • Data indicates a growing interest in discussing race and ethnicity data in HCI research in recent years, with a notable increase in 2021 conference papers.
• Limitations and Future Directions:
  • Limitations:
    1. The scope primarily focuses on U.S. racial and ethnic contexts, limiting the exploration of global differences.
    2. The survey sample size is relatively small and may not fully represent the opinions of the researcher community.
    3. The screening and analysis methods used may overlook papers that do not explicitly mention race information.
  • Future Directions:
    1. Expand the collection and reporting of race and ethnicity information in a global context.
    2. Investigate the reasons behind authors not collecting such information and design tools to overcome these barriers.
    3. Develop regionally adaptive racial classification systems and enhance qualitative research methods to explore the complexities of race and ethnicity data.