Surch: Enabling Structural Search and Comparison for Surgical Videos

Title of the Paper

Surch: Enabling Structural Search and Comparison for Surgical Videos

Paper Information

• Subject Area: Video retrieval and comparison, medical education, especially video-based surgical procedure learning
• Keywords: video learning, procedural knowledge representation, surgical learning, structured video search, cross-video interaction

Research Background and Problem

• Phenomena and Issues:

  • Videos are an effective medium for learning procedural knowledge (e.g., surgical skills), but there are still challenges in learning surgical procedures through videos:
    • Structured knowledge in surgical procedures (e.g., composition and sequence of steps) is difficult to access.
    • Existing platforms provide limited support for cross-video comparison and searching of surgical steps.
    • Users spend significant time searching for videos, often relying only on video titles or uploader information for retrieval.
  • Analyzing surgical videos poses challenges in semantic information extraction and structural pattern recognition, which often require manual annotation by domain experts, leading to high costs.

• Significance:

  • Semantic search and cross-video comparison functionalities for surgical videos can significantly enhance medical education, especially for beginners and clinical residents.
  • Providing a structured view of surgical steps can systematically improve students' understanding of the overall process and details of surgical procedures, thereby accelerating the learning curve.

• Research Motivation:

  • By designing smarter video search and comparison tools, large-scale surgical video data can be transformed into actionable learning resources.
  • Introducing intelligent search functions based on structural information to video learning can improve learning efficiency and user experience.

Solution

• Method/System Design:

  • A system named Surch is proposed to support structured search and video comparison for surgical video learning.
  • System Functions:
    1. Video Search Interface: Provides interactive semantic graphs to represent the procedural structure of surgical steps.
    2. Video Comparison Interface: Supports synchronized navigation across stages and comparison of multiple videos.
    3. Clustering Workflow: Automatically generates procedural graphs showing surgical structures, highlighting differences between videos with detailed paths (e.g., branches and loops).

• Innovations:

  • Employs clustering methods and weighted schemes to identify potential structural patterns in surgeries.
  • Designs customized analytical dimensions based on surgical steps and branches, including optional steps, step repetitions, and path branches.
  • Uses computer vision techniques (CNN-LSTM) to automatically annotate staged content in surgical videos, reducing the high cost of expert-dependent annotations.
  • Supports "graph-interactive filtering search" generated by clicking on surgical stages, allowing personalized video browsing.

• Technical Implementation:

  • Data Processing: Annotated a dataset containing 296 prostatectomy videos, converting surgical stages into sequential data processed by Convolutional Neural Networks (CNN) and Long Short-Term Memory networks (LSTM).
  • Weighted System: Developed a weighted PCA method (weights primarily consider local frequency characteristics and cross-video global characteristics) to further optimize clustering results.
  • Interface Design: Utilizes graphical search and comparison functionalities, enabling users to synchronize operations between surgical "pathway graphs" and "video playback."

Research Results

• Quantitative and Qualitative Evaluation:

  • The system's clustering results (Silhouette score = 0.82) indicate good classification clarity.
  • Compared to tests without vectorized analysis and weighting schemes, Surch achieves better clustering performance.
  • Automated stage detection accuracy for surgical videos reached 78.4%, with improved precision after integrating advanced machine learning techniques.

• User Study:

  • Conducted user research with 11 surgical residents of different levels.
    • Results showed significant improvements in video search (faster and more effective content viewing) and comparison efficiency.
    • The system enhanced beginners' understanding of the diversity of surgical techniques, improving their learning experience.
    • Junior residents (PGY1-3) showed significant efficiency gains, quickly grasping standardized surgical step processes.

• Limitations and Future Directions:

  • The current procedural graph labeling system lacks intuitiveness for advanced users; future work will integrate human-computer collaboration to generate more clinically relevant labels.
  • The current surgical videos cover a limited range of surgery types; further exploration is needed to generalize the system to broader video content (e.g., other surgical categories, non-medical procedural knowledge).
  • The system's adaptation to complex video semantic details and potential user needs in non-surgical domains requires further improvement and evaluation.

Conclusion

• Contributions:

  1. Provided a fully annotated dataset of prostatectomy videos (296 videos).
  2. Proposed a novel structured search and comparison interface.
  3. Conducted quantitative and user evaluations, demonstrating the system's value in surgical education.
  4. Released code and models as data resources for further academic and practical exploration.

• Research Impact:

  • Surch opens new directions for video learning applications, offering intelligent tools tailored to the practical needs of surgical education and complex task knowledge transfer.