Independent Validation of the Player Experience Inventory: Findings from a Large Set of Video Game Players

Title of the Paper

"Independent Validation of the Player Experience Inventory (PXI): Findings from a Large Sample of Video Game Players"

Paper Information

• Domain: Game User Research (GUR) in Human-Computer Interaction (HCI), focusing on measuring player experience
• Keywords: Player experience, Player Experience Inventory, game user research, questionnaire, survey scale, scale validation, measurement tools, psychometrics

Research Background and Problem

• Identified Issues or Challenges:

  • Measuring players' subjective experiences is a core method in game user research, but questionnaires designed for this purpose have rarely undergone thorough validation.
  • The Player Experience Inventory (PXI) has developed rapidly but lacks independent validation, with prior validation samples predominantly consisting of young male students, limiting representativeness.
  • The immersion construct in PXI may have issues, and previous studies have used small sample sizes, insufficient for robust statistical validation of the scale's quality.

• Significance:

  • Research on player experience is critical for game design and evaluation, requiring reliable and validated tools to ensure the credibility and applicability of findings.
  • As digital game design becomes increasingly personalized, evaluation metrics must maintain consistency across diverse samples and contexts.

• Motivation and Related Work:

  • To advance independent and large-scale validation of the PXI, addressing the limitations of small sample sizes and constrained validation procedures in prior research.
  • To explore PXI's theoretical model, optimize the psychometric quality of the scale, and provide stronger tool support for both academic and industry applications.
  • To evaluate the potential inclusion of an "enjoyment" construct in PXI and investigate the issue of simplifying PXI into a single score rather than a multidimensional construct.

Solution

• Proposed Methods or Solutions:

  • Conducting a pre-registered online study, recruiting 1,518 players, collecting data through PXI and related scales (e.g., PENS, AttrakDif, and IMI), and performing a comprehensive psychometric quality assessment.
  • Testing various theoretical models of PXI, including the original 10-factor model, models with additional successor factors ("functional" and "psychosocial"), and a model with a general PX factor.
  • Using Confirmatory Factor Analysis (CFA) to evaluate scale quality, analyzing reliability, validity, and the convergent and discriminant validity of the constructs.

• Innovative Aspects:

  • Independent validation of PXI, addressing the limitation that all prior validation studies were led by the original development team.
  • Demonstrating that PXI is applicable to a broader audience, including gender-balanced groups.
  • Analyzing dimensions such as "immersion," which have been identified as problematic, to promote deeper theoretical exploration and optimization.

• Implementation Steps and Techniques:

  1. Employing the "critical incident technique," where participants recall and describe a recent gaming experience to establish a meaningful connection with subsequent questionnaires.
  2. Applying various statistical techniques to the collected data, including Confirmatory Factor Analysis (CFA), reliability coefficients, and correlation analysis, to explore the validity of theoretical models.
  3. Comparing the model fit of different PXI structures and evaluating the feasibility of adding an "enjoyment" factor and a single overall player experience score.

Research Findings

• Specific Results:

  • PXI demonstrated good psychometric quality in a large sample, with validation results showing broad applicability.
  • The additional "enjoyment" factor performed well and could be included in PXI as a supplementary reference.
  • The study confirmed that PXI's theoretical model should adhere to the 10-factor model with separate constructs for each factor. The use of successor factors ("functional" and "psychosocial") or a general PX factor is not recommended.

• Advantages and Comparisons:

  • This study validated PXI's applicability more comprehensively than previous research, covering a broader range of age and gender groups.
  • PXI's multidimensional construct data provide significant value for academic research and precise comparisons between game versions in practice.
  • Addressed the issue of prior samples being overly concentrated on a single demographic (young male students), ensuring greater representativeness of the scale.

• Experimental and Evaluation Results:

  • CFA results indicated that the 10-factor model (without higher-order factors) achieved the best data fit.
  • PXI demonstrated overall good reliability, but the "immersion" construct performed poorly, with low inter-item variance, convergent validity, and discriminant validity.
  • The "immersion" construct showed some overlap with other factors (e.g., meaning and enjoyment), necessitating further theoretical differentiation.

• Limitations and Future Directions:

  • Limitations:
    • The study's reliance on the "critical incident" recall method may have influenced certain results (e.g., positive rating bias).
    • Data collection was conducted in an online environment, which may differ from field experiments.
  • Suggestions for future research:
    • Further validate PXI's applicability in real gaming scenarios, such as through laboratory testing or comparisons with improved game versions.
    • Explore the definition of the "immersion" construct and its relationship with other experience dimensions in greater depth.
    • Strengthen independent validation of the mini version of PXI (miniPXI).

This study provides an independently validated tool for game user research. PXI demonstrates significant theoretical and practical value but still requires addressing specific construct challenges, such as the definition and measurement of immersion.