Co-Designing Multimodal Systems for Accessible Asynchronous Dance Instruction

Paper Title

Co-Designing Multimodal Systems for Accessible Asynchronous Dance Instruction

Publication Info

• Topic area: Accessible asynchronous dance instruction for blind and low vision (BLV) learners.
• Keywords: Accessibility, asynchronous learning, blind and low vision, dance instruction, multimodal systems, haptics, sound cues, audio description, participatory design, feedback systems.

Background and Problem

• Problem / challenge: Existing online exercise videos rely on visual demonstrations, which exclude BLV learners. Audio descriptions (AD) alone are insufficient for conveying complex dance movements, timing, and expressiveness. Current accessible movement instruction systems do not scale well to asynchronous, multi-movement dance routines.
• Significance: Making dance instruction accessible to BLV learners can improve inclusivity in physical activities, enhance life satisfaction, and reduce barriers to remote learning.
• Motivation and related work: Prior research explored multimodal feedback (e.g., haptics, sound) for individual movements in sports and yoga but did not address long, complex routines in asynchronous settings. This paper builds on these efforts by focusing on asynchronous dance instruction systems for BLV learners.

Solution

• Proposed approach: Co-design workshops involving BLV learners, dance instructors, and experts in sound, haptics, and AD to create multimodal asynchronous dance instruction systems.
• Novelty:
  1. Development of multimodal movement vocabularies combining verbal, sound, and haptic cues.
  2. Structured learning phases emphasizing movement structure, timing, and expressiveness.
  3. Customizable systems tailored to skill levels, dance styles, and modality preferences.
  4. Design implications (DI1–DI14) for future asynchronous dance instruction systems.
• Procedure and key techniques:
  1. Conducted three in-person co-design workshops with 28 participants.
  2. Participants taught dance clips non-visually, designed multimodal systems, and presented prototypes.
  3. Prototyping tools included soundboards, 3D models, and tactile stickers.
  4. Data analysis involved thematic coding of workshop transcripts and survey responses.

Results

• Concrete findings:
  • Verbal descriptions should break down movements into body part actions, relations, and overall positioning.
  • Sound cues can convey rhythm, timing, and expressiveness, while haptics can guide body positioning and provide feedback.
  • Multimodal vocabularies and phased instruction improve learning efficiency and engagement.
• Advantage over baselines:
  • Combines modalities (verbal, sound, haptics) for richer, more precise instruction compared to AD alone.
  • Supports asynchronous learning without requiring co-located instructors.
• Experiments / evaluation:
  • Workshops included teaching and designing systems for five dance styles (e.g., Salsa, Contemporary, Hip-hop).
  • Systems were evaluated through participant feedback and re-enactments of proposed designs.
• Limitations and future work:
  • Limited focus on specific dance styles; improvisational styles were not addressed.
  • Low-fidelity prototyping tools constrained design possibilities.
  • Future work includes high-fidelity system development, empirical validation, and exploration of adaptive systems.

Summary

This paper addresses the challenges of making asynchronous dance instruction accessible to BLV learners by co-designing multimodal systems with BLV participants, dance instructors, and technical experts. Key contributions include multimodal vocabularies, phased instruction strategies, and customizable system designs. Findings highlight the importance of combining verbal, sound, and haptic cues to convey movement structure, timing, and expressiveness. Design implications (DI1–DI14) provide a roadmap for future systems, which could extend to other movement-based activities like martial arts or aerobics. Future work will focus on high-fidelity prototypes, adaptive systems, and broader applicability across dance styles and learner populations.