Operationalizing Perceptions of Agent Gender: Foundations and Guidelines

Paper Title

Operationalizing Perceptions of Agent Gender: Foundations and Guidelines

Publication Info

• Topic area: Human-agent interaction with a focus on gender perception and operationalization.
• Keywords: Agent gender, human-agent interaction, gender perception, operationalization, social construct, gender norms, gender stereotypes, gender measurement, human-computer interaction, gender theory.

Background and Problem

• Problem / challenge: There is no standardized approach to defining, measuring, or operationalizing perceptions of agent gender in human-agent interaction research. Many studies manipulate agent gender without measuring it, and the field lacks consistent terminology, theoretical grounding, and validated measures.
• Significance: Understanding agent gender perception is critical for addressing user attitudes, behaviors, and biases, including stereotyping and toxicity, in human-agent interactions. Without standardization, results are difficult to compare, limiting scientific progress and practical applications.
• Motivation and related work: Prior work has explored agent gender through cues like voice, body, and behavior, but operationalizations are inconsistent and often rely on binary gender models. Existing tools and theories from human gender studies are underutilized or inappropriately applied to agents. This paper aims to address these gaps by proposing a meta-level framework for operationalizing agent gender perceptions.

Solution

• Proposed approach: A meta-level framework for operationalizing perceptions of agent gender, based on a unifying theoretical model and a structured approach to conceptualization and methodology.
• Novelty:
  1. Development of a meta-level framework grounded in the theory of gender as a social structure.
  2. Systematic scoping review of 51 studies to map existing terms, theories, and measures.
  3. Guidelines for standardizing operationalization, including conceptualization, definition, indicators, and measurement.
  4. Emphasis on inclusivity and reflexivity in addressing diverse gender perceptions and biases.
• Procedure and key techniques:
  1. Conducted a systematic scoping review following PRISMA-ScR guidelines, analyzing 51 studies from 2014–2024.
  2. Identified gaps in definitions, measures, and theoretical grounding in existing research.
  3. Proposed a theoretical model extending Risman’s "gender as a social structure" to agent gender perceptions.
  4. Developed a four-step operationalization process: concept, definition, indicators, and measurement.

Results

• Concrete findings:
  • 54% of studies manipulated agent gender without measuring it.
  • 58 different measures were used across 51 studies, with little consistency.
  • Most studies relied on binary gender models, limiting the scope of understanding.
  • Gender-related terms and definitions were often vague or contradictory.
• Advantage over baselines:
  • The proposed framework provides a structured, theory-driven approach to operationalizing agent gender, addressing gaps in standardization and inclusivity.
  • It enables comparability across studies and supports the development of validated measures.
• Experiments / evaluation:
  • The scoping review analyzed studies from venues like CHI, IJSR, and HRI, focusing on gender-related goals, terms, operationalizations, and measures.
  • Key dimensions of agent gender manipulation included voice (27 studies), body (27 studies), and name (12 studies).
  • Measures included Likert scales, semantic differentials, and open-ended questions, but lacked standardization.
• Limitations and future work:
  • The review excluded non-English and non-peer-reviewed work, potentially limiting its scope.
  • Analysis was conducted by individual researchers, raising the risk of bias.
  • Future work should focus on validating measures, developing new tools, and addressing intersectionality in agent gender perceptions.

Summary

This paper addresses the lack of standardization in operationalizing perceptions of agent gender in human-agent interaction research. Through a systematic scoping review, it identifies gaps in definitions, measures, and theoretical grounding, and proposes a meta-level framework based on the theory of gender as a social structure. The framework includes a four-step process for conceptualization and methodology, emphasizing inclusivity and reflexivity. By adopting this framework, the research community can achieve greater consistency, enabling meta-analyses and advancing understanding of agent gender perceptions.