Smooth as - The Effects of Frame Rate Variation on Game Player Quality of Experience

Title of the Paper

The Effects of Frame Rate Variation on Game Player Quality of Experience

Paper Information

• Subject Area: Impact of frame rate variation on game player experience quality
• Keywords: Game player experience (QoE), frame rate variation, user study, computer games, frame time standard deviation, frame rate lower bound, algorithm optimization, graphics processing performance, cloud gaming, human-computer interaction

Research Background and Issues

• Identified Challenges or Issues:

  1. High average frame rates do not necessarily guarantee a good experience, as variations in frame display time can negatively impact quality of experience (QoE).
  2. The impact of frame rate stability on player experience has not been systematically quantified.
  3. Frame rate variations are common across different hardware and gaming conditions, but their specific effects on player experience remain underexplored.

• Significance: Frame rate variations affect players' visual smoothness and input responsiveness, significantly influencing in-game operational performance and overall enjoyment. This has important implications for hardware upgrade decisions and game development optimization.

• Research Motivation and Related Work: This study aims to fill the knowledge gap regarding the impact of frame rate variation on player QoE and provide an optimization framework for players, game developers, and hardware system designers. Related work has primarily focused on average frame rates in gaming and video experiences, with less attention given to frame rate variation and its effects on different game types and user interaction modes.

Solution

• Proposed Methods or Solutions: The authors evaluate the impact of frame rate variation on player QoE through user studies, designing an experimental framework to quantitatively measure QoE changes under different frame rate targets and variation conditions.

• Innovations:

  1. The first systematic study of the impact of frame rate variation on game player QoE, introducing frame time standard deviation and frame rate lower bound as QoE prediction metrics.
  2. Development of an experimental framework to artificially introduce frame rate variations by loading the CPU, testing their impact on user experience.
  3. Testing three different types of games to ensure the generalizability of the findings.

• Implementation Steps and Key Techniques:

  1. Game Selection: Includes Rocket League, Strange Brigade, and Valorant, covering different perspectives, visual scopes, and player input types.
  2. Framework Configuration: Games are run on high-performance computers, while Python scripts are used to create CPU loads to introduce frame rate variations.
  3. User Study Design: Randomized experimental conditions involving 33 participants, measuring QoE under two target frame rates (60Hz and 120Hz) and seven variation conditions for each frame rate.
  4. Analysis Tools: Presentmon tool is used to record frame times, and subjective QoE ratings are collected through user surveys.
  5. Statistics and Modeling: Linear prediction models based on frame time standard deviation and frame rate lower bound are established and their accuracy analyzed.

Research Findings

• Specific Findings:

  1. Average frame rate is insufficient for effectively predicting QoE, as it fails to account for the impact of frame rate variations.
  2. Frame time standard deviation is a valid predictor of QoE, though its predictive capability may vary across specific games.
  3. The 95% frame rate lower bound (i.e., the lowest 5% frame rate experienced by players) is a more effective and consistent QoE predictor.
  4. QoE remains acceptable when the 95% frame rate lower bound reaches 80% of the target frame rate.

• Advantages:

  1. Provides a unified frame rate lower bound model that does not require game-specific adjustments.
  2. Data demonstrates the widespread impact of frame rate variations, offering concrete optimization targets for game developers and hardware designers.
  3. Offers a framework for evaluating the impact of frame rate variations on user experience, which can be generalized to other game types or application scenarios.

• Experimental and Evaluation Results: Experiments reveal that frame time standard deviation predicts QoE well (overall linear model R² = 0.99), but the frame rate lower bound prediction model performs more consistently across individual games (R² > 0.82 for different games), outperforming the standard deviation method.

• Limitations and Future Directions:

  1. The current method tests only one hardware configuration; other hardware (e.g., GPU and storage devices) may exhibit different frame rate variation patterns.
  2. The player sample is concentrated on young male university students, which may not adequately represent other demographic groups.
  3. The range of games tested is limited; future research could expand to other game genres (e.g., MOBA, RTS).
  4. The frame rate variation model may be applicable to other human-computer interaction scenarios, such as virtual reality and video conferencing, and future studies could validate its applicability in these contexts.

This study provides theoretical and practical tools for improving game player experience through in-depth user experimental analysis, benefiting both academia and industry.