Music Creation by Example
Short online videos have become the dominating media on social platforms. However, finding suitable music to accompany videos can be a challenging task to some video creators, due to copyright constraints, limitations in search engines, and required audio-editing expertise. One possible solution to these problems is to use AI music generation. In this paper we present a user interface (UI) paradigm that allows users to input a song to an AI engine and then interactively regenerate and mix AI-generated music. To arrive at this design, we conducted user studies with a total of 104 video creators at several stages of our design and development process. User studies supported the effectiveness of our approach and provided valuable insights about human-AI interaction as well as the design and evaluation of mixed-initiative interfaces in creative practice.
Research Questions / Practical Problems
Question signals indexed for this paper.
- 75%
In a Silent Way: Communication Between AI and Improvising Musicians Beyond Sound
CHI '19· Generative AI (Text, Image, Music, Video) +1
- 75%
Understanding the Potentials and Limitations of Prompt-based Music Generative AI
CHI '25· Generative AI (Text, Image, Music, Video) +1
- 75%
Expressive Communication: Evaluating Developments in Generative Models and Steering Interfaces for Music Creation
IUI '22· Generative AI (Text, Image, Music, Video) +1
- 67%
MIMOSA: Human-AI Co-Creation of Computational Spatial Audio Effects on Videos
C&C '24· Generative AI (Text, Image, Music, Video) +2
- 60%
LoopMaker: Automatic Creation of Music Loops from Pre-recorded Music
CHI '18· Generative AI (Text, Image, Music, Video) +1
- 60%
On the Design of OLO Radio: Investigating Metadata as a Design Material
CHI '18· AI-Assisted Creative Writing +1
- 60%
Investigating Slowness as a Frame to Design Longer-Term Experiences with Personal Data: A Field Study of Olly
CHI '19· Generative AI (Text, Image, Music, Video) +1
- 60%
Giving Voice to Silent Data: Designing with Personal Music Listening History
CHI '20· Intelligent Voice Assistants (Alexa, Siri, etc.) +1
- 60%
Entangling Entanglement: A Diffractive Dialogue on HCI and Musical Interactions
CHI '24· Generative AI (Text, Image, Music, Video) +1
- 60%
Using Incongruous Genres to Explore Music Making with AI Generated Content
C&C '24· Generative AI (Text, Image, Music, Video) +2
Based on Jaccard similarity of research subtopics & professions (≥60%)