Remembering through Sound: Co-creating Sound-based Mementos together with People with Blindness

Title of the Paper

Remembering through Sound: Co-creating Sound-based Mementos with People with Blindness

Bibliographic Information

• Research Domain: Human-Computer Interaction (HCI) and design, focusing on sound-based memory aids and co-creative design.
• Keywords: memory, co-design, blindness, sound, audio memory, design research, collaboration

Research Background and Problem

• Research Questions and Challenges:
  • Most studies on sound-based memory focus on sighted individuals, rather than those with visual impairments.
  • People with blindness face unique requirements and challenges in capturing, recalling, and sharing memories through sound recordings.
  • Existing research primarily addresses how blind individuals overcome daily challenges (e.g., navigation, screen readers), but lacks in-depth exploration of how sound can enhance social connections, emotional reflection, and memory experiences.
• Research Significance:
  • Sound, as a primary sensory modality, holds significant value for blind individuals in capturing and sharing meaningful life moments.
  • Exploring how people with blindness use sound to construct richer personal and social memory experiences addresses a gap in current HCI research.
• Research Motivation and Related Work:
  • This study builds on the authors' previous work on memory experiences of blind individuals, emphasizing how sound-based memory can foster personalized and socialized recollection behaviors.
  • It proposes collaborative exploration with blind participants to investigate the possibilities of sound-based memory design, offering recommendations for future audio design and technology.

Solution

• Research Methods and Process:
  • Employing "Research through Design" and "Co-design" methodologies, the study engaged blind participants and their significant others in a 12-month multi-phase design process.
  • The design process consisted of three phases:
    1. Understanding Phase (Introductory Group Interviews): Establishing an initial understanding of participants' needs through audio documentaries and interviews.
    2. Exploration Phase (Participatory Field Activities): Investigating how participants create and use sound-based memories in their daily lives through sound-related activities.
    3. Co-design Phase (Co-design Workshop): Co-creating and implementing three types of sound-based mementos (sonic mementos) based on insights from the previous phases.
• Innovative Contributions:
  • Proposed three sound-based memento designs tailored to blind participants: encapsulation and condensation, augmentation and expansion, re-creation and re-imagination.
  • Incorporated sound narratives and storytelling tools, utilizing audio editing and online sound effect libraries to assist in memory reconstruction and emotional resonance.
• Design Methods:
  • Layered Sound Characteristics: Categorizing sound into foreground (focus sound), midground (supplementary sound), and background (ambient sound) for meaningful design.
  • Scenario Framework Construction: Introducing beginning and ending sound effects to form emotional narratives of memories.
  • Slow-paced Effects: Using slow pacing and extended sound effects to enhance auditory immersion and reflective space.

Research Outcomes

• Specific Results:
  • Three examples of sound-based mementos:
    1. Encapsulation and Condensation: Participants processed sound clips from martial arts classes, distilling key moments and adding narrative cues to make recordings more memory-triggering.
    2. Augmentation and Expansion: Participants enriched memory recordings by adding missing natural sounds and narrative extensions, such as reconstructing an island trip.
    3. Re-creation and Re-imagination: Using sound effects and storytelling to recreate an ordinary day from a child's growth period, evoking more vivid emotions and memories.
• Advantages Compared to Existing Solutions:
  • Developed sound design processes and techniques better suited to the needs of blind individuals, including low-vision tools, sound samples, and non-visual narrative methods.
  • Explored collaborative design models involving blind participants, balancing accessibility and participatory autonomy.
• Experimental Evaluation Results:
  • Participants reported that sound-based mementos not only helped them revisit past emotions and memories but also facilitated resonance and sharing with their significant others.
  • Multi-layered sound manipulation enhanced the authenticity, detail, and atmosphere of memory scenes in audio texts.
• Limitations and Future Directions:
  • Limitations:
    • The sample size of participants was relatively small, limiting generalizability across different social and cultural contexts.
    • The sound-based memory design process was time-consuming, with a learning curve for audio processing tools.
    • Although feedback loops were established, improving frequency and formats of participant feedback remains an area for enhancement.
  • Future Directions:
    • Expand participant diversity to explore cross-cultural and cross-community sound design needs.
    • Develop more automated and precise sound processing support tools (e.g., AI-based sound highlight tagging).
    • Advance the translation of research outcomes into user experience products, exploring long-term usage effects and impacts.

Conclusion

Through this study, the authors successfully proposed a collaborative design model for sound-based memory, offering new perspectives and practical tools to enhance individual memory experiences and social sharing for blind individuals. This work also extends the boundaries of research and practice in sound design within the HCI domain.