D-Twins: Your Digital Twin Designed for Real-Time Boredom Intervention

Research Background and Problem

• Identified Problems or Challenges: Boredom is pervasive in automated environments, especially during low-stimulation tasks such as monitoring data or participating in prolonged online meetings. Traditional conversational agents lack personalization and emotional understanding, relying solely on predefined tasks or limited dialogue corpora, which hampers their ability to accurately identify and respond to users' emotions, thus limiting their effectiveness in alleviating boredom.
• Why This Problem is Important: Boredom not only affects individual experiences but also reduces work efficiency, increases operational errors, and may even lead to safety risks, such as decreased attention in autonomous vehicle monitoring. Addressing boredom can enhance user alertness and productivity, improving the human-computer interaction experience in automated environments.
• Research Motivation and Related Work: Affective AI and large language models (LLMs) offer advanced emotional recognition capabilities and flexible interaction methods. By integrating user physiological data, personalized AI agents can be created. These emotionally resonant "digital twins" can explore real-time solutions to mitigate boredom.

Solution

• Proposed Method or Solution: The authors propose an LLM-based affective AI agent called "D-Twins," which can reflect users' emotional states and personalities in real time. It detects boredom through user expressions and EEG (electroencephalogram) data and provides personalized interventions.
• Innovative Aspects of the Solution:
  1. Digital Reflection: D-Twins creates a personalized AI agent through natural language processing, capable of embodying users' personality traits.
  2. Emotional State Alignment: Shares emotional states with users, enhancing interaction experiences through "emotional resonance."
  3. Real-Time Personalized Interventions: Integrates EEG data to detect boredom in real time and dynamically adjusts intervention strategies based on the detection results.
• Implementation Steps and Key Technologies:
  1. Data Collection and Modeling: User dialogue data is collected using the Wizard-of-Oz method to train a personalized large language model.
  2. Boredom Detection Model: An SVM-based boredom classifier is trained using EEG data from 50 participants, achieving a maximum accuracy of 87.2%.
  3. System Integration: The LLM model is integrated with the EEG detection functionality into D-Twins, which performs boredom alleviation and emotional activation functions through a dynamic interaction framework.

Research Outcomes

• Specific Achievements: Experiments demonstrate that D-Twins effectively detects users' boredom and alleviates it through personalized dialogue. This real-time intervention not only reduces boredom but also increases user engagement and trust.
• Advantages Compared to Existing Solutions:
  1. High Personalization: Digital twins reflect users' linguistic, behavioral, and personality traits.
  2. Emotional Resonance: Enhances interaction experience by sharing users' emotional states (e.g., boredom).
  3. Real-Time Flexibility: Dynamically adjusts model responses using EEG data.
• Experimental or Evaluation Results:
  1. Subjective evaluations show a significant reduction in users' boredom levels (Wilcoxon test p < 0.01), particularly in dimensions such as attention, time perception, and low arousal.
  2. EEG data indicates that users' brainwave patterns (e.g., θ and α waves) in the intervention state are closer to non-boredom states compared to the non-intervention state, demonstrating D-Twins' effectiveness in improving user emotions.
  3. Users expressed satisfaction with the linguistic and behavioral similarities between D-Twins and themselves, which deepened interaction and emotional connection.
• Limitations and Future Directions:
  1. Task Design: Current tasks are simple and do not measure the impact of boredom on actual work performance. Future studies should explore more complex scenarios to quantify intervention effects.
  2. Multimodal Interaction: D-Twins relies solely on text-based dialogue and lacks multimodal information (e.g., visual, auditory), resulting in lower social presence. Future research could explore the integration of video and audio.
  3. Long-Term Application: Boredom detection and intervention effectiveness have been validated primarily in experimental settings. Further studies are needed to extend the research to broader user groups or real-world scenarios.

Conclusion

This study successfully designed and validated the personalized AI agent D-Twins, which combines LLM and EEG technologies to detect boredom in real time and provide personalized interventions. By sharing emotional states with users and engaging in personalized dialogue, D-Twins not only significantly alleviates user boredom but also demonstrates the potential of emotional resonance, offering a new pathway to enhance human-computer interaction experiences. This research provides valuable insights for the future development of personalized and real-time interventions in AI applications within automated environments, while also highlighting the challenges and opportunities in designing highly human-like AI agents.