Movement as Medium: Personalisation of Instruments for Inclusive Creative Expression in Disability-Led Performance

Paper Title

Movement as Medium: Personalisation of Instruments for Inclusive Creative Expression in Disability-Led Performance

Publication Info

• Topic area: Personalised gestural instruments for inclusive creative expression in performance and community contexts.
• Keywords: Personalisation, inclusive design, gestural instruments, disability-led performance, creative expression, HCI, participatory design, research-through-design, practice-based research, capability approach.

Background and Problem

• Problem / challenge: Existing gestural systems often rely on generalised mappings that constrain performers to generic gestures, failing to accommodate individual movement styles and creative intentions. This is particularly limiting for performers with physical disabilities.
• Significance: Personalisation is essential to ensure technologies support individual creative practices and expand expressive possibilities, particularly for those with physical disabilities. This work addresses the need for inclusive, expressive interaction design.
• Motivation and related work: Prior research in Accessible Digital Musical Instruments (ADMIs) often lacks engagement with disability models and overemphasizes corrective perspectives. This paper builds on participatory and co-design methods to explore personalisation as a methodological approach, addressing gaps in ecologically validated, performance-led accounts of personalised instrument design.

Solution

• Proposed approach: A personalised gestural instrument, the "interactive paintbrush," was co-designed with a professional performer with physical disability and tested in professional and community contexts.
• Novelty:
  1. Articulation of personalisation as a design methodology for inclusive interaction.
  2. Development of a handheld, metaphor-rich interactive paintbrush for real-time multimodal control.
  3. Ecological validation of the instrument in professional performance and participatory workshops.
  4. Transfer of personalisation strategies to diverse users and contexts.
• Procedure and key techniques:
  1. Iterative co-design with a professional performer, integrating practice-based research (PbR) and research-through-design (RtD).
  2. Development of a handheld instrument with custom hardware, software, and digitally fabricated enclosures.
  3. Ecological validation through rehearsals, live performances, and workshops with young participants with physical disabilities.
  4. Real-time calibration and integration of personal creative materials to support diverse movement capabilities.

Results

• Concrete findings:
  • The interactive paintbrush enabled real-time multimodal control of sound, visuals, and lighting, supporting expressive and inclusive performance.
  • Automatic calibration allowed adaptation to diverse movement ranges, enabling independent art-making for participants with limited mobility.
  • Integration of personal creative materials (e.g., paintings, photographs) enhanced creative agency and engagement.
• Advantage over baselines:
  • Greater alignment with individual movement styles and creative intentions compared to generalised ADMIs.
  • Enabled professional-level performance and independent art-making for participants with physical disabilities.
• Experiments / evaluation:
  • Professional performance of Conduit Bodies (five-show season) provided ecological validation under live conditions.
  • Four workshops with six young participants demonstrated scalability and adaptability of personalisation strategies.
• Limitations and future work:
  • Limited generalisability due to a single-performer case study and a small workshop cohort.
  • Future work will explore additional input/output modalities, alternative physical designs, and expanded mentorship opportunities.

Summary

This paper presents a personalised gestural instrument, the interactive paintbrush, co-designed with a professional performer with physical disability. The instrument enabled real-time multimodal control of sound, visuals, and lighting, supporting expressive and inclusive performance in both professional and community contexts. Ecological validation through rehearsals, live performances, and workshops demonstrated the value of personalisation in accommodating diverse movement capabilities and creative imperatives. The research advances HCI by framing personalisation as a design methodology that prioritises individual expression, creative agency, and inclusion, offering a model for designing technologies that support diverse bodies and practices.