A11yBoard: Making Digital Artboards Accessible to Blind and Low-Vision Users

Title of the Paper

A11yBoard: Making Digital Artboards Accessible to Blind and Low-Vision Users

Paper Information

• Research Area: Accessibility Computing, Human-Computer Interaction
• Keywords: Accessibility Technology, Blind and Low-Vision Users, Digital Artboards, Multimodal Interaction, Cross-Device Interaction, Accessible Content Creation, Risk-Free Exploration

Research Background and Problem

• Problem or Challenge: Current digital artboards (e.g., Microsoft PowerPoint, Google Slides) are difficult for Blind and Low-Vision (BLV) users to access. These tools rely heavily on visual cues and cannot effectively convey two-dimensional spatial information through linear methods such as existing screen readers. As a result, resources and tools for accessible content creation are relatively scarce.
• Significance: With the increasing prevalence of artboards in educational and professional contexts, the lack of tools enabling BLV users to easily navigate and create content limits their participation in design and content creation.
• Research Motivation and Related Work: While previous research has improved BLV users' access to content such as charts and web pages, there is insufficient research on content creation tools (e.g., layout design, wireframe development, poster creation). Traditional screen readers provide limited support for two-dimensional spaces (such as artboards), particularly in handling challenges like relative positioning of objects, layering order, and property manipulation.

Solution

• Method or Solution:
  • Introduced A11yBoard, an interactive multimodal system that enables BLV users to access and create digital artboards.
  • Combines a web-based desktop application with a mobile touchscreen device, the latter used for safe exploration of two-dimensional spaces.
  • Incorporates voice recognition, non-speech audio, and keyboard-based commands as input/output methods.
• Innovations:
  1. Cross-Device Interaction: Provides BLV users with multiple feedback and interaction channels through the combination of desktop and touchscreen devices.
  2. Risk-Free Exploration: Users can explore spatial positions and object information on the touchscreen via tonal and voice feedback.
  3. Separation of Modes: Separates object explanation from editing operations, reducing cognitive load during exploration.
  4. Multimodal Interaction: Combines touch, gestures, audio, and keyboard search functionalities.
• Implementation Steps and Technology:
  1. Built artboard functionality on the desktop using DrawerJS (an open-source drawing tool).
  2. Developed a synchronized mobile application supporting gesture-based exploration.
  3. Introduced "voice commands" and a "universal command line" feature to enable users to complete tasks using natural language.
  4. Provided real-time audio and voice feedback to reduce uncertainty during operations.

Research Outcomes

• Specific Results:
  • Significant improvement in users' ability to accurately identify the position, color, size, and relationships of objects on the artboard during interpretation tasks.
  • Users successfully created, aligned, and modified objects on digital artboards during generation tasks.
  • Multimodal interaction strategies (e.g., touchscreen exploration and keyboard commands) proved more efficient than traditional screen readers.
• Comparison with Existing Solutions:
  • A11yBoard offers users enhanced spatial awareness, addressing the limitations of screen readers in two-dimensional interactions.
  • Compared to traditional screen readers, task completion rates significantly improved (62 interpretation tasks fully successful, 96% of generation tasks partially or fully successful).
• Experimental and Evaluation Results:
  • NASA TLX subjective workload assessments indicated that users found A11yBoard to have low mental, physical, and time demands, high efficiency, and minimal frustration.
  • Two-phase user studies (including iterative preliminary testing and formal task experiments) validated the practicality of A11yBoard.
• Limitations and Future Directions:
  • Limitations:
    • The current system is based on DrawerJS, offering fewer features compared to mainstream artboard tools (e.g., PowerPoint).
    • Task experiments cannot fully simulate the complexity of real-world educational/professional scenarios.
  • Future Directions:
    • Add customization features (e.g., unit conversion, adjustable levels of detail in output).
    • Explore integration possibilities with mainstream tools (e.g., VoiceOver, PowerPoint).
    • Extend A11yBoard technology to more design tools (e.g., video editing, image drawing tools) and other 2D information spaces.

Through this research, A11yBoard not only validates innovative interaction concepts in a specific domain but also advances the development of accessible content creation platforms.