What Do We See in Them? Identifying Dimensions of Partner Models for Speech Interfaces Using a Psycholexical Approach

Title of the Paper

What Do We See in Them? Identifying Dimensions of Partner Models for Speech Interfaces Using a Psycholexical Approach

Paper Information

• Subject Area: Human-Computer Interaction (HCI) and Psychology, focusing on users' psychological perception models of speech interaction systems.
• Keywords: Partner models, mental models, speech interfaces, psycholexical approach, human-machine dialogue, psychometrics

Research Background and Problem

• Identified Problems or Challenges:

  • Existing research has recognized the importance of "partner models" in speech interaction systems, but the intrinsic dimensions of this concept remain unclear.
  • There is a lack of in-depth research on how users evaluate the linguistic functionality and social attributes of speech interaction systems.

• Significance:

  • Understanding partner models of speech systems can help design interactive interfaces that better meet user needs.
  • It can guide future research on linguistic interaction behaviors and optimize user experience.

• Research Motivation and Related Work:

  • Previous studies on partner models mainly focused on users' assumptions about human and machine conversational abilities and their impact, but lacked specific dimensional details.
  • Related work has revealed the influence of human-like characteristics (e.g., vocal expression, conversational strategies) in speech system design on user cognition.
  • The authors argue that the conceptualization in existing research is too broad and needs more detailed dimensions to enrich the partner model framework.

Solution

• Method or Solution:

  • Employing the psycholexical approach to extract dimensions of users' perceptions of partner models from language.
  • Using Principal Component Analysis (PCA) to cluster word pairs and identify key dimensions of partner models.

• Innovations:

  • The first systematic quantification of users' partner models for speech interaction systems.
  • Proposed three core dimensions of partner models: Competence and Reliability, Human-likeness, and Cognitive Flexibility.

• Implementation Steps and Key Techniques:

  1. Phase 1: Using the Repertory Grid Technique (RGT) to prompt users to generate word pairs related to speech systems, resulting in 246 unique word pairs.
  2. Phase 2: Compiling relevant word pairs from existing survey questionnaires, generating an additional 155 word pairs.
  3. Filtering word pairs to remove redundancy, irrelevance, or overly ambiguous terms, retaining 51 word pairs.
  4. Online Survey Study: Users evaluated their past experiences with speech systems using the 51 word pairs, with 356 participants.
  5. Principal Component Analysis (PCA): PCA was applied to analyze user responses, clustering word pairs and determining core dimensions.

Research Findings

• Specific Findings:

  • Partner models consist of three primary dimensions:
    1. Competence and Reliability: Involves evaluations of system capabilities, accuracy, and dependability.
    2. Human-likeness: Involves perceptions of human-like attributes in speech systems (e.g., warmth, sociability).
    3. Cognitive Flexibility: Reflects the system's interactivity, adaptability, and spontaneity.

• Advantages:

  • Provides a concrete framework for understanding users' mental models of speech systems, offering theoretical support for speech interaction system design and conversational behavior prediction.
  • Expands the conceptualization of partner models, laying the groundwork for systematic future research.

• Experimental or Evaluation Results:

  • PCA analysis retained 37 relevant word pairs, and after removing redundant pairs, extracted three significant dimensions.
  • Empirical analysis showed that the three dimensions collectively explained 49% of the data variance.

• Limitations and Future Directions:

  • Limitations:
    • The study primarily focused on disembodied speech systems, lacking research on interactions involving robots or virtual agents with physical attributes.
    • Reflective questionnaires were used, which might obscure direct psychological reactions during specific interactions.
  • Future Directions:
    • Extend partner models to more interaction scenarios and devices.
    • Develop a fully validated measurement scale for assessing users' partner models.
    • Explore potential causal relationships and dynamic changes between partner model dimensions.

Conclusion

This study, through the psycholexical approach and multi-method analysis, has for the first time clarified the core dimensions of users' psychological partner models for speech interaction systems—Competence and Reliability, Human-likeness, and Cognitive Flexibility. This has significant implications for expanding the applicability of speech systems and improving user experience in the HCI field, while providing theoretical and methodological tools for future research.