Signals of Success and Struggle: Early Prediction and Physiological Signatures of Human Performance across Task Complexity

Paper Title

Signals of Success and Struggle: Early Prediction and Physiological Signatures of Human Performance across Task Complexity

Publication Info

• Topic area: Early prediction of human task performance using physiological signals in escalating complexity environments.
• Keywords: Human performance, physiological signals, ocular features, cardiac features, task complexity, prediction models, multimodal fusion, eye tracking, heart rate variability, affective experience.

Background and Problem

• Problem / challenge: Existing performance prediction models primarily rely on behavioral metrics, which lack insight into the cognitive and physiological processes underlying task performance. The potential of ocular and cardiac signals for early prediction under escalating task complexity remains underexplored.
• Significance: Anticipating performance outcomes can enable timely interventions, improve task continuity, and enhance user experience in high-stakes or engagement-driven environments.
• Motivation and related work: Prior work has demonstrated the feasibility of performance forecasting using behavioral and physiological metrics, but these approaches often focus on short-term predictions or momentary states. Ocular and cardiac signals have been used to study workload, attention, and stress, but their combined predictive potential across escalating task complexity is unclear.

Solution

• Proposed approach: A multimodal prediction framework using early ocular and cardiac signals to forecast later performance outcomes in a progressively complex game environment.
• Novelty:
  1. Demonstrates the predictive value of early-session ocular and cardiac signals for later performance under escalating complexity.
  2. Identifies systematic differences in visual and autonomic patterns between high- and low-performing groups.
  3. Integrates subjective affective experiences to contextualize physiological patterns.
• Procedure and key techniques:
  • Conducted a within-subject experiment with 35 participants playing a deck-building game (Slay the Spire).
  • Collected ocular data (e.g., saccades, gaze distribution) using Tobii Pro Fusion and cardiac data (e.g., heart rate, HRV) using Empatica EmbracePlus.
  • Extracted features from low-complexity sessions to predict performance in high-complexity sessions using supervised classifiers (CatBoost, XGBoost, Linear SVM) and decision-level fusion.
  • Evaluated models using leave-one-subject-out (LOSO) cross-validation.

Results

• Concrete findings:
  • The fused ocular–cardiac model achieved a balanced accuracy of 0.86, outperforming unimodal models (ocular-only: 0.83; cardiac-only: 0.70).
  • Key predictive features included gaze allocation to task-relevant areas, saccade dynamics, and heart rate.
  • High-performing participants exhibited targeted gaze, efficient visual sampling, and stable cardiac activation as task complexity increased.
• Advantage over baselines:
  • The fused model improved classification of low-performing participants compared to ocular-only and cardiac-only models.
  • Decision-level fusion enhanced performance by leveraging complementary information from both modalities.
• Experiments / evaluation:
  • Participants completed two game sessions (low and high complexity), with physiological signals recorded and self-reported affective ratings collected.
  • Models were evaluated using metrics such as balanced accuracy, macro-F1, and Matthews Correlation Coefficient (MCC).
• Limitations and future work:
  • Limited to a single game context; findings may not generalize to other interactive tasks.
  • Cardiac data acquisition was constrained by the low sampling rate of the wearable device, limiting HRV reliability.
  • Lack of a pre-task baseline to separate task-evoked responses from individual traits.
  • Small sample size; future studies should include larger and more diverse cohorts.

Summary

This study demonstrates that early ocular and cardiac signals can predict later performance outcomes under escalating task complexity in a game environment. The fused model combining ocular and cardiac features achieved the highest predictive accuracy, with key contributions from gaze allocation, saccade dynamics, and heart rate. High-performing participants showed strategic visual behavior and stable autonomic activation, while low-performing participants exhibited less efficient patterns. Subjective affective ratings revealed differences in valence and dominance between groups. These findings highlight the potential of multimodal physiological signals for proactive performance prediction and suggest applications in adaptive systems, training, and safety-critical tasks. Future work will explore generalizability across contexts, refine cardiac measures, and investigate individual differences in failure trajectories.