AVscript: Accessible Video Editing with Audio-Visual Scripts

Title of the Paper

AVscript: Accessible Video Editing with Audio-Visual Scripts

Paper Information

• Domain: Human-Computer Interaction, Accessibility Technology, Video Editing
• Keywords: Video, Video Editing Tools, Accessible Design, Text Editing, Blind Creators, Human-Computer Interaction

Research Background and Problem

• What problems or challenges did the authors identify?

  • Existing mainstream video editing tools are highly inconvenient for visually impaired users (including blind and low-vision users, abbreviated as BLV), primarily because these tools rely on visual elements (such as timelines and interface icons) for operation.
  • BLV creators face four major challenges in video editing: 1) Difficulty in accessing the visual content of videos; 2) Inability to evaluate the visual quality of videos, such as blurry or poorly lit segments; 3) Tool menus are not screen reader-friendly; 4) Difficulty in efficiently navigating or locating video content non-linearly.
  • As a result, many BLV creators rely on sighted collaborators to complete their tasks or upload unedited videos directly, limiting their independence and creative potential.

• Why is this problem important?

  • As more BLV creators attempt to share their content on platforms like YouTube, developing suitable video editing tools for this group has significant social implications.
  • Improving tool accessibility can enhance BLV creators' independence, confidence, and content quality.

• Research Motivation and Related Work

  • This study builds on existing accessible creation tools (e.g., text and presentation editing tools) and accessible video playback technologies (e.g., audio descriptions) to explore, for the first time, how to improve video editing accessibility for BLV users.
  • The authors examined the current state of BLV creators and video editing tutorials, proposing an innovative audio-visual script editing model.

Solution

• What methods or solutions did the authors propose?

  • The authors designed and developed AVscript, an accessible text-based video editing tool that integrates visual content, visual errors (e.g., blurriness, poor lighting), and video narrative scripts into a text script to assist BLV users in editing videos.
  • The tool incorporates scene descriptions, high-level visual navigation, visual error detection, and text-video synchronization operations.

• What are the innovative aspects of the solution?

  • Integration of text and video: Provides video content and visual quality descriptions in a single text interface, enabling screen reader users to quickly access information.
  • Automated visual error detection: Identifies issues such as blurriness, occlusion, and lighting problems, clearly marking potential cut points for BLV creators.
  • Combination of global and local navigation: Supports non-linear browsing through scene outlines and detailed scripts.
  • Multi-modal support: Tailored features for BLV users, such as audio notifications and quick jump functionality without manual navigation.

• What are the implementation steps and key technologies used?

  1. Script generation: Transcribes audio into text while detecting visual scenes and aligning them with speech.
  2. Visual error detection: Utilizes computer vision technologies (such as Detic and OpenCV) to detect issues like poor lighting and blurriness.
  3. Navigation support: Develops a navigable outline with scene descriptions and supports search-based quick jumps.
  4. Editing features: Enables operations like deleting, trimming, and adjusting the speed of video content.

Research Outcomes

• What specific outcomes were achieved?

  • Reduced cognitive load: Experiments (N=12) showed that using AVscript significantly reduced the mental workload of BLV users during video editing, while improving efficiency.
  • Support for diverse video creation: In exploratory studies, participants expressed that AVscript inspired them to try creating a wider variety of video types and styles, while reducing their reliance on sighted collaborators.
  • Successful user cases: Participants successfully and efficiently edited their own videos in real-life scenarios, using scene descriptions and error prompts to identify precise cut points, which boosted their confidence in creation.

• What advantages does it have compared to existing solutions?

  • Bridges the "accessibility gap" between BLV creators and existing video editing tools.
  • Significantly optimizes user operations by avoiding reliance on deep menus and icons, greatly improving scene editing efficiency for BLV creators.

• What were the experimental or evaluation results?

  • Compared to traditional tools used by participants, editing with AVscript significantly reduced cognitive load (66% improvement in ratings) and time pressure (57% improvement in ratings).
  • During the experiments, users heavily relied on AVscript's automated visual error prompts to ensure visual quality.

• Limitations and Future Directions

  • Limitations:
    • Accuracy in describing complex video content (e.g., scenes with multiple simultaneous tasks) needs improvement.
    • Current visual error detection (e.g., identifying blurriness in key objects) may miss some details.
    • Initial use requires a certain adaptation period.
  • Future Directions:
    • Add severity levels and more detailed descriptions for visual errors (e.g., distinguishing between minor and significant blurriness).
    • Enhance multi-track editing features, such as support for titles and inserted animations.
    • Incorporate AI-generated auxiliary descriptions to improve accuracy and scene information.

Through this research, AVscript provides BLV creators with a novel possibility for video editing, demonstrating a trend towards efficient and accessible video creation.