"Do I Really Need This?": Illuminating Challenges in Integrating Computational Training Tools in Esports Coaching

Paper Title

"Do I Really Need This?": Illuminating Challenges in Integrating Computational Training Tools in Esports Coaching

Publication Info

• Topic area: Integration of computational training tools (CTTs) in esports coaching.
• Keywords: Esports, computational training tools, translational HCI, League of Legends, coaching, mechanical skills, player performance, technology adoption, qualitative study, quantitative trends.

Background and Problem

• Problem / challenge: Computational training tools (CTTs) developed in research contexts are rarely adopted in real-world esports coaching due to challenges in integration, skepticism, and lack of actionable insights.
• Significance: Esports is a growing domain with increasing demand for tools to enhance player performance, but the lack of adoption of research-driven tools limits their practical impact.
• Motivation and related work: Prior work has developed CTTs for esports, focusing on performance evaluation and skill improvement. However, these tools often remain confined to laboratory settings, with limited exploration of real-world integration challenges. Similar barriers have been observed in other domains like education and healthcare, where cost, time, and entrenched workflows hinder adoption.

Solution

• Proposed approach: Development and deployment of MySkills, a bespoke CTT for League of Legends, designed to assess and visualize players’ mechanical skills, and integrated into a professional esports academy.
• Novelty:
  1. Deployment of a research-driven CTT in a real-world esports coaching environment for three months.
  2. Mixed-method evaluation combining quantitative analysis of skill trends and qualitative interviews with stakeholders.
  3. Identification of barriers to adoption and alignment of findings with translational HCI frameworks.
• Procedure and key techniques:
  • Development of MySkills based on prior research on mechanical skill indices (e.g., GMS, W-index, LoL-specific skills).
  • Integration into a South Korean League of Legends academy, including onboarding coaches and students.
  • Monthly reports generated for players, visualizing mechanical skill metrics and trends.
  • Two rounds of semi-structured interviews with coaches and students to capture perceptions and experiences.
  • Quantitative analysis of skill trends using Interrupted Time Series (ITS) analysis.

Results

• Concrete findings:
  • Quantitative analysis showed slow but steady improvement in mechanical skills during the deployment period.
  • ITS analysis revealed significant slope changes in skill metrics for some participants, suggesting potential benefits of the tool.
  • 403 performance reports were generated, but engagement with the tool declined over time.
• Advantage over baselines: While skill improvements were observed, skepticism and lack of engagement limited the perceived advantages over traditional coaching methods.
• Experiments / evaluation:
  • Participants: 2 coaches and 8 students, supplemented by insights from the academy’s head coach.
  • Metrics: Mechanical skill indices (e.g., GMS, W-index, LoL-specific skills).
  • Methods: Qualitative thematic analysis of interviews; ITS analysis of skill trends.
• Limitations and future work:
  • Limited sample size (10 participants, all male) and context (single academy).
  • Misalignment between tool capabilities and stakeholder expectations.
  • Future work should explore diverse contexts, larger samples, and improved communication of tool affordances.

Summary

This study developed and deployed MySkills, a computational training tool for League of Legends, in a professional esports academy to examine the challenges of real-world integration. Quantitative analysis revealed skill improvements, but qualitative interviews highlighted barriers such as skepticism, lack of time, and difficulty interpreting outputs. Stakeholders perceived the tool as novel but overwhelming, with limited long-term engagement. These findings align with broader translational HCI challenges and suggest the need for clearer communication of tool benefits, tailored integration strategies, and infrastructure to support esports coaching. Future work should address these barriers to enhance the adoption and impact of CTTs in esports.