Psychometric Properties of the User Experience Questionnaire (UEQ)

Title of the Paper

Psychometric Properties of the User Experience Questionnaire (UEQ)

Bibliographic Information

• Subject Area: Psychometric properties of user experience (UX) measurement and evaluation tools
• Keywords: User experience, psychometric properties, UEQ, HCI evaluation methods, usability, reliability, user questionnaire, UX theory, design model, UX research framework

Research Background and Issues

• Identified Problems or Challenges:

  • User experience research originated as a challenge to traditional usability studies, which were deemed overly focused on task efficiency and work effectiveness.
  • The current scope of user experience extends beyond goal achievement, encompassing complex layers such as satisfaction of stimulation, personal growth, and interpersonal communication needs during user interaction. Its measurement requires objectivity, reliability, and validity.
  • The UEQ questionnaire has been widely used for years, but studies evaluating its psychometric properties (e.g., reliability and validity) remain relatively scarce.

• Significance:

  • The reliability and validity of standardized UX measurement questionnaires (such as UEQ) can provide guidance for academic research and practical applications.
  • This helps in understanding the relationship between users and products, pointing out areas for improvement in product development.

• Research Motivation and Related Work:

  • Previous studies have shown that the UEQ questionnaire can effectively measure usability and user experience, but they were based on small samples or insufficient statistical data.
  • This study aims to evaluate the psychometric properties of UEQ based on a larger sample cluster, covering theoretical model validation, reliability assessment, and experimental sensitivity testing of the questionnaire.

Solution

• Proposed Methods or Solutions:
  • For the first time, factor analysis is used to evaluate the theoretical model of UEQ (based on experimental data from 1,211 participants and evaluations of 23 products).
  • Correlation analysis is conducted between UEQ subscales and the System Usability Scale (SUS) to validate the questionnaire's discriminant and convergent validity.
  • Reproducibility of the UEQ scales across different products is evaluated, and the precision of the questionnaire is further validated by altering product characteristics in experimental designs.
• Innovations:
  • A new factor structure model is proposed, based on two main factors (pragmatic and hedonic qualities), which provides a more stable explanation for the UX questionnaire compared to the original six-factor classification.
  • Experimental validation is conducted to explore the questionnaire's sensitivity to changes in pragmatic and hedonic quality of user experience.
• Implementation Steps and Key Techniques:
  1. Questionnaire Completion and Data Collection:
     • Data from 1,211 participants are collected through laboratory, field, and online studies, covering various product types.
  2. Factor Analysis and Model Adjustment:
     • Principal Component Analysis (PCA) is used to evaluate the original six-factor structure and explore a model combining two factors (pragmatic and hedonic qualities).
  3. Reliability and Validity Assessment:
     • Cronbach's Alpha is used to assess internal consistency of the scales, and correlations between the questionnaire and usability scales are calculated.
  4. Experimental Design:
     • Four versions of a mobile app (Plain, Plain+Tutorial, Guided, Guided+Tutorial) are used to test the sensitivity of the UEQ questionnaire to pragmatic and hedonic quality changes.

Research Outcomes

• Specific Findings:
  • The reliability of the original six UEQ scales ranges from acceptable to good (α = 0.77 to α = 0.92).
  • Factor analysis indicates high correlations among the original six factors, and two main factors (pragmatic and hedonic qualities) better explain the data.
  • Experimental data show that UEQ is sensitive to changes in the pragmatic and hedonic qualities of products (e.g., interface complexity, introduction of non-task-related elements).
• Comparison with Existing Solutions and Advantages:
  • The new two-factor model structure not only improves reliability (pragmatic α = 0.92, hedonic α = 0.88) but also enhances the fit of the theoretical model.
  • Provides evaluation data for UEQ across different product characteristics, making it more applicable for practical guidance.
• Experimental or Evaluation Results:
  • Significant correlations are observed between the "Attractiveness" dimension and other elements within the scales ("Attractiveness" and pragmatic factor r=0.82; "Attractiveness" and hedonic factor r=0.65).
  • Experiments reveal that users prefer designs with simple functionality combined with hedonic elements, while avoiding complex and restrictive interfaces.
• Limitations and Future Directions:
  • The current study focuses on the UX measurement tool itself and does not compare it with other measurement instruments. Future research should aim to construct comprehensive models.
  • The complexity of the product samples is relatively limited; further studies could include more complex products to enhance ecological validity.

Summary and Conclusion

• This study demonstrates the reliability and validity of UEQ in measuring the pragmatic and hedonic qualities as well as the overall attractiveness of products.
• The research suggests interpreting UEQ data using the two-factor model in research contexts, while exercising caution when interpreting the six-factor results in industrial applications.
• Further refinement of the UEQ is needed to cover more detailed dimensions of user experience, and future work should advance more systematic models combining theory and practice.