CoKnowledge: Supporting Assimilation of Time-synced Collective Knowledge in Online Science Videos

Research Background and Issues

• Problems and Challenges:
  The authors propose that Danmaku (a type of scene-aligned, time-synchronized scrolling commentary) can expand the content of science popularization videos through "collective knowledge." However, the chaotic nature of these comments often makes it difficult for viewers to effectively absorb the embedded knowledge, especially in knowledge-intensive science videos. Major issues include varying comment quality, a large amount of repetitive content, and unclear information structure. Additionally, the interaction between video and comments often leads to distractions.

• Significance:
  Danmaku has been shown to have great potential in knowledge co-construction, enriching video content and mitigating the cognitive bias of uploaders. However, the current commenting experience does not effectively support viewers in absorbing and understanding this collective knowledge. This limits the practical value of Danmaku in science communication.

• Research Motivation and Related Work:
  Although previous studies have explored the emotional expression and content interaction of Danmaku, they mainly focus on enhancing user engagement and video analysis, with less attention paid to how viewers can better understand the knowledge potential of Danmaku itself. Therefore, the authors aim to fill this gap by improving the presentation and processing of Danmaku to help viewers comprehensively and effectively absorb the collective knowledge in science videos.

Solution

• Methods and System:
  The authors propose an interactive system called CoKnowledge, which supports the absorption of collective knowledge in science videos through the following modules:

  1. NLP Processing Pipeline: Utilizing natural language processing (NLP) techniques to filter, classify, and cluster Danmaku comments, enhancing information density and revealing their underlying knowledge structure.
  2. User Interface Design: Providing three modes (Overview Mode, Focus Mode, and Exploration Mode) to meet users' knowledge needs at different stages of video viewing:
     • Overview Mode: Offers a high-level progress bar and knowledge flow visualization to quickly locate key content.
     • Focus Mode: Enlarges the video window, significantly reduces floating comments, and provides a more focused viewing experience.
     • Exploration Mode: Displays a Knowledge Graph to allow users to deeply analyze specific video segments.
  3. Sidebar Support: Includes a subtitle-Danmaku list, AI-generated explanations, and related comments, providing on-demand detailed analysis.

• Technical Innovations:

  1. NLP Pipeline: Leveraging the Llama-2 model combined with LoRA adaptation to classify Danmaku, automatically categorizing comments into five major information types (e.g., explanatory comments, supplementary knowledge) and optimizing their display.
  2. Multi-layered Knowledge Organization: Using clustering algorithms (e.g., DBSCAN) to enhance the clarity and readability of Danmaku information while semantically linking comments to video content.

• Implementation Steps:

  1. Conducting content analysis on 35 videos and 28,088 Danmaku comments to build a training dataset and classification standards.
  2. Designing an automated processing pipeline to filter and classify comments and match them to video content based on semantics and timeline.
  3. Developing the CoKnowledge system, integrating layered display and mode-switching functionalities to support viewers' knowledge acquisition in different contexts.

Research Outcomes

• Specific Results:

  1. CoKnowledge significantly enhanced users' understanding and retention of collective knowledge, improving overall test scores by 56.1% compared to traditional interfaces, with a particularly notable improvement in the Danmaku section (117.8%).
  2. The system clarified knowledge structures and associations, increasing information density and reducing repetitive content.
  3. Compared to traditional Danmaku displays, user experience improved significantly, including higher knowledge absorption efficiency, stronger immersion, and greater interactivity.

• Advantages Over Existing Solutions:
  CoKnowledge retains the original interactivity and collaborative knowledge-building capabilities of Danmaku while significantly reducing information interference and cognitive load through intelligent processing and layered display. This design is supported by experimental results and user interviews, providing a practical direction for enhancing the educational potential of knowledge-intensive videos.

• Experiments and Evaluation:
  A comparative experiment (N=24) was conducted to analyze the system's knowledge transmission effectiveness, user interaction patterns, and overall usability. Users demonstrated significantly higher confidence and efficiency in knowledge acquisition under the CoKnowledge condition, with task load comparable to the baseline system.

• Limitations and Future Directions:

  1. The data primarily comes from the Bilibili platform, which may introduce platform-specific biases.
  2. Participants were predominantly younger users, potentially affecting generalizability.
  3. Some users felt that filtering out non-knowledge-related Danmaku reduced entertainment value. Future work could explore how to balance entertainment and knowledge absorption.
  4. The system's applicability needs further validation, especially in educational videos and other user-generated content platforms.

Conclusion

CoKnowledge successfully leverages intelligent technology to address the issue of information overload in science video Danmaku, enhancing the absorption of collective knowledge. By introducing automated processing methods and interactive user interface design, the system provides a novel tool for online science communication and establishes a design pathway for knowledge co-construction in video interactions.