ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators

Paper Title

ADCanvas: Accessible and Conversational Audio Description Authoring for Blind and Low Vision Creators

Publication Info

• Topic area: Accessible audio description (AD) tools for blind and low vision (BLV) creators.
• Keywords: Audio description, accessibility, blind and low vision, multimodal models, conversational agents, human-AI collaboration, screen reader, creative tools, video authoring, inclusive design.

Background and Problem

• Problem / challenge: Existing audio description (AD) tools are primarily designed for sighted users, relying on visual interfaces like timelines and waveform editors. These tools are inaccessible to BLV creators, forcing them to rely on sighted collaborators and limiting their creative autonomy.
• Significance: AD is critical for BLV audiences to access visual media, and enabling BLV creators to independently produce AD enhances inclusivity and representation in media production.
• Motivation and related work: Prior research has explored semi-automated AD tools and community-driven platforms, but these primarily serve sighted creators or consumers. Multimodal large language models (MLLMs) offer potential for accessible AD creation, but their integration into non-visual workflows for BLV creators remains underexplored.

Solution

• Proposed approach: ADCanvas, a multimodal AD authoring tool that combines conversational AI, keyboard-based navigation, and a screen-reader-accessible editor to enable BLV creators to independently create and refine AD scripts.
• Novelty:
  1. Introduction of a screen-reader-accessible AD authoring tool that integrates conversational interaction, visual question answering (VQA), and timestamped script editing.
  2. Empirical findings from a user study with 12 BLV participants, highlighting human-AI co-creation practices and challenges.
  3. Design implications for accessible creative tools, emphasizing configurability, precision, and non-visual interaction.
• Procedure and key techniques:
  1. Users interact with a conversational agent to ask visual questions, generate summaries, and draft AD scripts.
  2. A WebVTT-based editor allows timestamped script editing with screen reader support.
  3. Hotkeys enable non-visual navigation and playback control.
  4. Real-time text-to-speech (TTS) narration supports quality control and synchronization with video content.

Results

• Concrete findings:
  • 10 out of 12 participants rated ADCanvas as "very useful" (7/7), and 11 participants rated their likelihood of using it as 7/7.
  • The conversational agent accurately responded to 89.1% of prompts, with 10.9% containing errors or inaccuracies.
• Advantage over baselines:
  • Enables independent AD authoring for BLV creators without reliance on sighted collaborators.
  • Combines VQA, script generation, and editing in a single accessible workflow.
• Experiments / evaluation:
  • Conducted with 12 BLV participants using three short videos representing different AD authoring challenges (quality control, scaffolded authoring, and free-form authoring).
  • Data collected through think-aloud protocols, interaction logs, and post-session interviews.
• Limitations and future work:
  • Limited to short, dialogue-light videos; future work should include dialogue-rich and longer videos.
  • Does not support advanced features like waveform inspection or fine-grained timing alignment.
  • Future research should explore long-term deployment, collaborative workflows, and scaling across genres and production roles.

Summary

ADCanvas is a novel, accessible AD authoring tool that empowers BLV creators to independently create, refine, and control AD scripts through conversational AI, screen-reader compatibility, and keyboard-based navigation. A user study with 12 BLV participants demonstrated its effectiveness in supporting creative workflows, with high ratings for usefulness and likelihood of adoption. While the system successfully addresses accessibility barriers in AD creation, future work should focus on advanced features, customization, and broader deployment to enhance its applicability across diverse contexts. ADCanvas represents a significant step toward inclusive media production, enabling BLV creators to exercise creative ownership and independence.