A Shared Look: Detecting Deepfakes with Inter-Subject Neural Synchrony

Paper Title

A Shared Look: Detecting Deepfakes with Inter-Subject Neural Synchrony

Publication Info

• Topic area: Neuro-cognitive approaches to deepfake detection
• Keywords: Deepfake detection, inter-subject neural synchrony, EEG hyperscanning, generative AI, Hyper-FusionNet, cognitive load, predictive coding, human-centered AI, explainable systems, digital trust

Background and Problem

• Problem / challenge: Existing deepfake detection methods rely heavily on transient digital artifacts, which are increasingly ineffective against advanced generative AI techniques like image-to-video (I2V) forgeries. These methods also lack interpretability and alignment with human perception, limiting user trust.
• Significance: The proliferation of deepfakes threatens social trust, individual reputations, and national security. Developing robust, interpretable detection systems is critical for combating misinformation and enhancing digital literacy.
• Motivation and related work: Prior work has focused on machine-centric detection of pixel-level artifacts or human-in-the-loop systems, both of which have limitations. Emerging neuroscience research suggests that the human brain exhibits distinct neural responses to synthetic media, providing a potential foundation for a new detection paradigm.

Solution

• Proposed approach: The study introduces a neuro-cognitive method based on Inter-Subject Neural Synchrony (ISNS), using EEG hyperscanning to detect deepfakes by analyzing shared neural responses between paired viewers.
• Novelty:
  1. Development of a novel detection paradigm grounded in shared human neural responses rather than transient digital artifacts.
  2. Introduction of the Hyper-FusionNet model, which integrates multi-view neural synchrony metrics and topological priors for robust classification.
  3. Empirical demonstration of "AI-induced hyper-synchrony" as a biomarker for synthetic content.
  4. Validation of ISNS biomarkers for both deepfake detection and emotion recognition, showcasing their versatility.
• Procedure and key techniques:
  • EEG hyperscanning of 15 participant pairs viewing real and I2V-generated videos.
  • Analysis of inter-brain synchrony metrics (ISC, PLV, Coherence) across five frequency bands.
  • Development of Hyper-FusionNet, a dual-stream architecture combining ResNet-based feature extraction and a graph neural network (GNN) branch.
  • Evaluation using a 15-fold Leave-One-Pair-Out Cross-Validation (LOPO-CV) strategy.

Results

• Concrete findings:
  • ISNS patterns exhibited systematic differences between real and fake videos, with "AI-induced hyper-synchrony" observed in Beta and Gamma bands for fake content.
  • Hyper-FusionNet achieved 89.23% accuracy (95% CI: [84.54%, 93.92%]) in deepfake detection.
  • ISNS biomarkers also enabled emotion recognition with 79.49% accuracy (95% CI: [73.68%, 85.30%]).
• Advantage over baselines: The proposed model outperformed single-modality baselines and traditional classifiers, demonstrating the importance of multi-dimensional fusion and topological priors.
• Experiments / evaluation:
  • Participants viewed 36 videos (real and I2V-generated) under a 2x2x3 factorial design.
  • EEG data were processed to extract inter-brain synchrony metrics, which were used to train and evaluate Hyper-FusionNet.
  • Results were validated using LOPO-CV to ensure generalizability across unseen participant pairs.
• Limitations and future work:
  • Limited ecological validity due to controlled laboratory settings.
  • Small sample size (15 pairs) typical of hyperscanning studies; larger datasets are needed for further validation.
  • Generalizability across diverse generative AI architectures remains to be tested.
  • Challenges in scaling the method for real-world applications, including hardware usability and computational efficiency.

Summary

This study introduces a novel neuro-cognitive approach for deepfake detection, leveraging Inter-Subject Neural Synchrony (ISNS) as a biomarker for authenticity. Using EEG hyperscanning, the authors identified "AI-induced hyper-synchrony" as a robust neural signature of synthetic content and developed the Hyper-FusionNet model, achieving 89.23% accuracy in detecting I2V deepfakes. The findings also demonstrated the versatility of ISNS biomarkers in emotion recognition, highlighting their potential for broader applications. By grounding detection in shared human perception, this work offers a pathway toward more trustworthy, explainable, and human-centered content verification systems. Future research will focus on scaling the approach for real-world deployment and validating its robustness across diverse contexts and generative models.