The Subconscious Director: Dynamically Personalizing Videos Using Gaze Data

Title of the Paper

The Subconscious Director: Dynamically Personalizing Videos Using Gaze Data

Bibliographic Information

• Subject Areas: Visual User Interaction, Adaptive Video Technology, Eye Movement Data Analysis
• Keywords: Eye tracking, adaptive media, gaze-based systems, preference prediction, user engagement

Research Background and Problem Statement

• What issues or challenges did the authors identify? As entertainment habits evolve, watching TV or videos has increasingly become a casual companion activity, making it harder for filmmakers to capture viewers' attention. Traditional interactive films require viewers to actively choose the storyline, which contradicts the audience's desire for effortless entertainment.

• Why is this problem significant? In the modern entertainment landscape, effectively capturing and maintaining viewers' attention is a critical task for media creators. By removing barriers to active user participation and providing personalized video experiences, it is possible to enhance user engagement and entertainment outcomes.

• Research Motivation and Related Work:

  • Netflix's interactive film Black Mirror: Bandersnatch allows viewers to actively choose storyline branches, but this approach is more akin to video games than traditional filmmaking.
  • Using gaze data (eye movement data) as a basis for inferring viewer preferences has been applied in psychology and market research, but research has yet to explore how this data can be dynamically applied to personalized video content creation.

Solution

• What methods or solutions did the authors propose? This paper introduces a dynamic video personalization system based on gaze data, called "eyeDirect." The system records users' gaze direction and duration while watching videos to infer preferences and adjust the storyline in real time.

• What are the innovative aspects of this solution?

  • Utilizing gaze data as the foundation for preference prediction eliminates the need for active user choices, significantly reducing interaction complexity.
  • Enables dynamic adjustment of movie storylines, allowing viewers to subconsciously become the "director" of the film.
  • Integrates machine learning algorithms to improve the accuracy of preference prediction while exploring differences between simple majority voting and machine learning-based methods.

• What are the implementation steps and key technologies used?

  1. Gaze Data Collection: Real-time recording of viewers' gaze direction and duration using screen-based eye trackers.
  2. Feature Extraction: Extracting nine key gaze-related features, such as total number of gaze points, average gaze duration, and maximum consecutive gaze duration.
  3. Preference Prediction: Testing two prediction methods—simple majority voting and machine learning-based prediction—using historical data to train the model.
  4. Branch Selection: Selecting the next scene based on prediction results to dynamically adjust the storyline.
  5. User Experimentation: Analyzing the impact of personalized videos on user experience through control and experimental groups.

Research Findings

• What specific results were achieved?

  • Dynamic personalized videos significantly improved users' focused attention and involvement.
  • Machine learning models achieved significantly higher preference prediction accuracy compared to simple voting methods, with a maximum accuracy of 78.18%.

• What advantages does it have compared to existing solutions?

  • Compared to interactive films requiring manual storyline selection, eyeDirect reduces user interaction burden.
  • Compared to systems based on simple gaze data analysis, the integration of machine learning filters out interference from salient video elements, enabling more accurate preference inference.

• What were the experimental or evaluation results?

  • User experiments involving 175 participants revealed:
    • Personalized videos using machine learning predictions significantly enhanced focused attention and involvement (p-value < 0.05).
    • The impact of personalized videos on "novelty" was not significant, remaining similar to non-personalized videos.

• Limitations and Future Directions:

  • Limitations:
    • The system relies on high-precision eye tracking devices, and accuracy may decline with low-cost equipment or in real-world environments.
    • The system has only been tested under laboratory conditions, and its performance in complex everyday scenarios remains unverified.
    • The long-term impact of personalized videos on user experience (e.g., repeated viewing) is still unclear.
  • Future Directions:
    • Investigate the integration of physiological sensor data (e.g., heart rate or facial expressions) to enhance the robustness of preference prediction.
    • Develop optimized versions of the system for mobile devices and low-cost hardware.
    • Explore the potential of personalized videos in education, advertising, and other specific application domains.