T1GER: An Instructional Re-Design of a Cyber Range Exercise in a Commercial Security Operations Center

Paper Title

T1GER: An Instructional Re-Design of a Cyber Range Exercise in a Commercial Security Operations Center

Publication Info

• Topic area: Instructional design for cybersecurity training in Security Operations Centers (SOCs).
• Keywords: Cyber range, instructional design, SOC training, Tier 1 analysts, learning management system, cognitive load, learning motivation, cybersecurity education, randomized controlled trial, threat analysis.

Background and Problem

• Problem / challenge: Many cyber range exercises prioritize technical realism but lack effective instructional design, leading to cognitive overload and suboptimal learning outcomes for trainees, especially novice Tier 1 analysts.
• Significance: Effective training is critical to address the cybersecurity workforce shortage and ensure SOCs can detect and mitigate cyberattacks efficiently.
• Motivation and related work: Prior research has proposed instructional enhancements for cyber ranges, such as feedback and structured scenarios, but these efforts often focus on isolated elements and lack empirical validation in real-world SOC contexts. This study seeks to address these gaps by integrating and evaluating comprehensive instructional design principles in a commercial SOC setting.

Solution

• Proposed approach: The Tier 1 Cyber Range Exercise Instructional Redesign (t1ger), an augmented Learning Management System (LMS) for cyber range exercises, integrates structured feedback, procedural scaffolding, and competitive elements.
• Novelty:
  1. Embedding task-specific hints and scaffolding directly into the LMS to reduce trainer dependence.
  2. Providing immediate, task-level feedback with explanations to enhance learning accuracy.
  3. Introducing gamification through a scoring system and leaderboard to increase engagement.
  4. Retaining the technical authenticity of the original cyber range while improving instructional design.
• Procedure and key techniques:
  1. Analyze gaps in the existing LMS used by the collaborating SOC.
  2. Implement instructional design principles (e.g., scaffolding, feedback, competition) into the t1ger LMS.
  3. Conduct a randomized controlled trial with 144 participants to compare t1ger against the original LMS.
  4. Measure learning experience, knowledge outcomes, and training efficiency using validated psychometric tools and performance metrics.

Results

• Concrete findings:
  • Participants using t1ger reported higher learning motivation (M = 3.71 vs. 3.33, p = 0.001) and lower cognitive load (M = 3.49 vs. 3.01, p = 0.001).
  • Training completion rates were higher in the t1ger group (86% vs. 69%, p = 0.024).
  • Participants in the t1ger group completed the training faster (64.92 vs. 73.37 minutes, p < 0.001).
  • Knowledge outcomes were comparable between groups across all categories.
• Advantage over baselines:
  • Improved learning experience (motivation and cognitive load).
  • Increased training efficiency (completion rates and time).
  • Reduced reliance on trainers without compromising learning outcomes.
• Experiments / evaluation:
  • Randomized controlled trial with 144 participants from two European universities.
  • Metrics included learning motivation, cognitive load, knowledge outcomes (TAP, SIEM, scenario-specific, and general alert investigation), training completion, and time.
• Limitations and future work:
  • Limited to a single SOC scenario (log4j vulnerability); generalizability to other SOCs and scenarios needs further exploration.
  • Long-term retention of learning outcomes was not assessed.
  • Future work could explore applying instructional design principles to other SOC tasks, generating tailored scenarios, and integrating AI-based sparring partners.

Summary

The t1ger LMS redesign for cyber range exercises significantly improved learning motivation, reduced cognitive load, and increased training efficiency for Tier 1 SOC analysts without compromising knowledge outcomes. By embedding instructional design principles such as scaffolding, feedback, and gamification, t1ger reduced reliance on trainers and enhanced the scalability and reproducibility of SOC training. These findings highlight the potential of interdisciplinary approaches to transform cybersecurity education and address workforce challenges in SOCs. Future research could expand the application of these principles to other SOC tasks and explore long-term learning impacts.