A Design Space for Live Music Agents

Paper Title

A Design Space for Live Music Agents

Publication Info

• Topic area: Interdisciplinary framework for analyzing and designing intelligent systems in live music performance.
• Keywords: Live music agents, human-AI interaction, music improvisation, design space, AI music systems, real-time music, computational creativity, music technology, interdisciplinary research, ecosystem analysis.

Background and Problem

• Problem / challenge: The development of live music agents has been fragmented across HCI, AI, and computer music communities, leading to limited integration of insights, methods, and tools. This fragmentation hinders the creation of systems that balance technical, musical, and interactional needs.
• Significance: Live music agents have the potential to transform musical creativity and performance by enabling real-time human-AI collaboration. Addressing the fragmentation can unlock new possibilities for innovation and responsible development.
• Motivation and related work: Early systems focused on rule-based accompaniment and improvisation, while recent advances in AI have enabled more sophisticated capabilities. However, gaps remain in understanding interdisciplinary design principles, societal implications, and technical challenges such as latency and adaptation.

Solution

• Proposed approach: A comprehensive design space framework categorizing live music agents across four aspects: Usage Context, Interaction, Technology, and Ecosystem.
• Novelty:
  1. Systematic analysis of 184 systems from academic literature and videos, identifying 31 dimensions and 165 codes.
  2. Integration of interdisciplinary perspectives from HCI, AI, and computer music.
  3. Public release of annotated systems as a living artifact for community exploration and refinement.
• Procedure and key techniques:
  • Conducted systematic literature review and video analysis to sample systems.
  • Iterative coding process to identify dimensions and codes.
  • Developed an interactive visualizer to enable exploration and application of the design space.

Results

• Concrete findings:
  • 80% of systems focus on live performance, while 85% target musicians as users.
  • Latency is emphasized in 85% of systems, but error handling and tempo adaptability are underexplored.
  • Generative AI methods are emerging but remain a small fraction (6%) of systems.
  • Ecosystem dimensions, such as policy considerations and economic consequences, are notably underrepresented.
• Advantage over baselines: The design space provides a structured framework for interdisciplinary analysis, revealing trends, gaps, and opportunities that were previously fragmented or overlooked.
• Experiments / evaluation:
  • Case studies of systems like jam_bot, Shimon, and ReaLchords illustrate practical applications of the design space.
  • Interactive visualizer supports analysis, comparison, and design generation.
• Limitations and future work:
  • Limited double-coding of the corpus may introduce interpretation errors.
  • Dataset may underrepresent non-English publications and commercial tools.
  • Future work includes expanding the corpus, refining dimensions, and integrating community-driven contributions.

Summary

This paper introduces a design space for live music agents, analyzing 184 systems to identify 31 dimensions across Usage Context, Interaction, Technology, and Ecosystem aspects. The framework consolidates interdisciplinary insights and highlights trends, gaps, and opportunities for innovation. Case studies demonstrate its utility in analyzing existing systems and generating new designs. Despite limitations in coding and dataset scope, the design space serves as a robust tool for advancing responsible and creative development of live music agents.