Capturing Team Cognition: A Multimodal Dataset for Adaptive Collaborative Interfaces

Paper Title

Capturing Team Cognition: A Multimodal Dataset for Adaptive Collaborative Interfaces

Publication Info

• Topic area: Multimodal sensing and brain-computer interfaces for distributed team collaboration.
• Keywords: Team cognition, EEG hyperscanning, brain-computer interfaces, distributed collaboration, multimodal dataset, neural synchrony, creative problem solving, adaptive systems, team processes, real-time monitoring.

Background and Problem

• Problem / challenge: Existing collaboration tools lack visibility into cognitive and affective states shaping team performance, particularly in geographically dispersed settings. Traditional assessments rely on subjective self-reports or manual annotations, which are biased, disruptive, and impractical for large-scale applications. Few datasets capture neural and behavioral signals from distributed teams.
• Significance: Real-time monitoring of team dynamics could predict breakdowns, enhance coordination, and improve creative outcomes, addressing critical challenges in human factors and organizational psychology.
• Motivation and related work: Advances in physiological computing and BCIs have focused on individual users in controlled settings, leaving gaps in understanding multi-person neural dynamics in distributed contexts. Hyperscanning studies have primarily examined co-located dyads, with limited exploration of larger groups or geographically dispersed teams.

Solution

• Proposed approach: A multimodal dataset capturing synchronized EEG, audio transcripts, screen interactions, and behavioral annotations from distributed teams engaged in creative collaboration across continents.
• Novelty:
  1. Demonstrating the feasibility of capturing synchronized EEG and behavioral data in distributed teams.
  2. Introducing a publicly available multimodal dataset for studying team cognition in naturalistic settings.
  3. Establishing methods to link neural signals to team processes and performance.
  4. Providing a foundation for adaptive interfaces responding to team-level cognitive and behavioral states.
• Procedure and key techniques:
  • Teams of 3–4 participants collaborated remotely using EEG headsets, Zoom, and shared digital tools (Miro whiteboard, ChatGPT).
  • Data acquisition included EEG signals, video/audio recordings, and behavioral annotations.
  • Neural metrics (e.g., Task Engagement Index, Task Load Index, mutual information, recurrence rate) and behavioral indicators (team processes, task performance) were derived and analyzed.

Results

• Concrete findings:
  • Team performance scores ranged from 49.5 to 93.5 out of 100 (M = 74.8, SD = 14.1).
  • Neural measures showed modest correlations with task performance (e.g., DE NL3 synchrony r = .59, mutual information r = .23).
  • Mutual information correlated strongly with action processes (r = .71), while recurrence rate negatively correlated with interpersonal processes (r = −.72).
• Advantage over baselines:
  • Dataset enables modeling of team cognition in geographically distributed settings, addressing gaps in prior hyperscanning studies limited to co-located dyads.
  • Provides robust infrastructure for real-time monitoring and adaptive system development.
• Experiments / evaluation:
  • Teams designed virtual escape rooms, integrating narrative, puzzle variety, and user experience considerations.
  • EEG metrics were analyzed alongside behavioral data using correlation and regression techniques.
• Limitations and future work:
  • Vulnerability to EEG-specific artifacts and confounds (e.g., volume conduction, shared stimuli effects).
  • Need for additional controls (e.g., surrogate data, spatial filtering) and exploration of non-verbal interactions.
  • Future directions include integrating visual analytics platforms and extending multimodal datasets for broader applications.

Summary

This paper introduces a multimodal dataset capturing EEG and behavioral data from distributed teams engaged in creative collaboration, enabling real-time monitoring of team cognition. The dataset includes synchronized neural and behavioral signals, providing insights into cognitive states and team dynamics. Analyses reveal modest correlations between neural measures and team performance, supporting the feasibility of adaptive systems that respond to team-level signals. By addressing gaps in prior hyperscanning studies and providing publicly available resources, this work lays the foundation for developing brain-computer interfaces that enhance distributed teamwork.