Keeping Everyone in the Game: Bringing Ability-Inclusive Family Co-Play to Unmodified Console Games

Paper Title

Keeping Everyone in the Game: Bringing Ability-Inclusive Family Co-Play to Unmodified Console Games

Publication Info

• Topic area: Ability-inclusive gaming for mixed-skill family co-play.
• Keywords: Ability-inclusive play, family co-play, mixed-age gaming, controller skills, computer vision, input assistance, lifespan study, intergenerational gaming, PartyAssist, unmodified games.

Background and Problem

• Problem / challenge: Mixed-age family gaming often exposes skill gaps, leading to frustration, exclusion, and reduced enjoyment. Existing solutions like Dynamic Difficulty Adjustment (DDA) or role-based balancing require integration into game development, leaving many unmodified games inaccessible to mixed-ability groups.
• Significance: Addressing skill gaps in family gaming can enhance family closeness, connectedness, and intergenerational interaction, making gaming more inclusive and enjoyable for all participants.
• Motivation and related work: Prior work has explored game-authored balancing, input aids, and vision-based augmentation, but these approaches are limited to specific games or contexts. This paper aims to develop a platform-agnostic solution that works with unmodified games, bridging the gap in accessibility for existing titles.

Solution

• Proposed approach: PartyAssist, a real-time computer-vision-to-input assistance system that provides subtle controller-level assistance for unmodified console games.
• Novelty:
  1. Open-source Controller Skills Benchmark assessing six core skills across the lifespan (ages 6–90).
  2. First lifespan study revealing skill-specific performance gaps between children and older adults.
  3. Real-time input assistance system for unmodified games using computer vision and input injection.
  4. Empirical evidence and design guidelines for ability-inclusive family co-play.
• Procedure and key techniques:
  1. Develop a Controller Skills Benchmark to measure six key gamepad skills (e.g., reaction time, thumbstick control).
  2. Implement PartyAssist, which uses computer vision (YOLOv8) to analyze game state and inject micro-assists into controller inputs.
  3. Conduct a feasibility study with 16 mixed-skill dyads (n=32) to evaluate the system's impact on performance and user experience.

Results

• Concrete findings:
  • Children (<10) showed significant performance improvements in games like Granite Getaway (+48.4% survival time) and Defuse or Lose (+200% hit count).
  • Older adults (>50) also improved significantly in Granite Getaway (+112.5% survival time) and Defuse or Lose (+300% hit count).
  • Children reported higher enjoyment with assistance (+1.19 average rating), while older adults had mixed enjoyment (-0.44 average rating).
• Advantage over baselines:
  • PartyAssist enabled performance gains without requiring game modifications.
  • Assistance compensated for specific skill deficits (e.g., thumbstick control for older adults).
• Experiments / evaluation:
  • Lifespan study with 80 participants (ages 6–90) quantified skill gaps using six controller tasks.
  • Feasibility study with 16 dyads tested PartyAssist on five Super Mario Party Jamboree mini-games, measuring performance (e.g., survival time, success rate) and subjective experience.
• Limitations and future work:
  • Small sample size and single-platform focus limit generalizability.
  • Assistance policies were game-specific; future work aims to develop generalized and adaptive modules.
  • Future studies will explore in-home settings, longitudinal effects, and comparisons with existing game-assist features.

Summary

This paper introduces PartyAssist, a vision-to-input assistance system that bridges skill gaps in family gaming by providing real-time controller-level support for unmodified console games. A lifespan study (n=80) identified skill-specific performance gaps, and a feasibility study (n=32) demonstrated that PartyAssist improved performance for children and older adults in mixed-skill dyads. While children generally enjoyed the assistance, older adults had mixed reactions, highlighting the importance of social context and perceived agency. The findings provide a foundation for designing inclusive gaming systems that adapt to user abilities and social dynamics, with future work focusing on generalization, personalization, and broader evaluation.