Supporting Rhythm Activities of Deaf Children using Music-Sensory-Substitution Systems

Rhythm is the first musical concept deaf people learn in music classes. However, hearing loss limits the amount of information that allows a deaf person to evaluate his or her performance and stay in sync with other musicians. In this paper, we investigated how a visual and vibrotactile music-sensory-substitution device, MuSS-Bits++, affects rhythm discrimination, reproduction, and expressivity of deaf people. We conducted a controlled study with 11 deaf children and found that most participants felt more confident wearing the device in vibration mode even when it did not objectively improve their accuracy. Furthermore, we studied how MuSS-Bits++ can be used in music classes at deaf schools and what challenges and opportunities arise in such a setting. Based on these studies, we discuss insights and future directions that support the design and development of music-sensory-substitution systems for music making.