Engaging with Nature Sounds & Citizen Science by Designing for Creative & Contextual Audio Encounters

Title of the Paper

Promoting Interaction with Natural Sounds and Citizen Science through Creative and Contextual Audio Interaction Design

Paper Information

• Subject Areas: Human-Computer Interaction (HCI), Citizen Science, Ecological Conservation, Audio Interaction Design
• Keywords: Natural Sounds, Audio, Birds, Citizen Science, Creativity, Emotion, Play, Purposeful Play, Learning, Game Design, Participatory Design, Sound, Wildlife

Research Background and Issues

• Problems or Challenges:

  1. The health of ecosystems is closely related to the sounds of wildlife, yet natural sounds are often overlooked in daily life.
  2. Current technologies and tools are insufficient for helping people learn and identify sounds in audio recordings, leading to gaps in ecological research and restoration, particularly concerning endangered bird species (e.g., Eastern Bristlebird).
  3. There has been limited exploration into creating engaging interactive experiences that encourage users to learn and understand natural sounds.
  4. The calls of certain bird species in nature remain under-researched, with high variability (e.g., Eastern Bristlebird), posing challenges for AI-based automatic detection.

• Significance: Understanding and protecting natural sounds are crucial for ecosystem restoration and wildlife conservation, especially in addressing the current environmental crisis. Public participation can accelerate efforts to protect endangered species.

• Research Motivation and Related Work:

  1. Earlier studies have explored designing interactive installations to spark interest in nature, but the complexity of visualizing audio (e.g., spectrograms) remains a challenge.
  2. Existing research by ornithologists and environmental audio recording experts highlights the importance of combining visual observation with sound.
  3. Designing appealing audio interaction experiences can not only attract attention but also aid species identification in citizen science and ecological conservation.

Solution

• Proposed Solution: The authors designed a prototype gamified activity called Bristle Whistle Challenge, aimed at fostering curiosity about natural sounds and supporting wildlife conservation within the framework of citizen science through interactive audio experiences.

• Innovations:

  1. Combining gamification with free-form exploratory activities, the design integrates highly contextual, creative, and multisensory audio interaction.
  2. Introducing a conversational agent (Bob Bristlebird) that provides emotional support and guidance through casual dialogue.
  3. Users learn and identify bird calls through multisensory interaction involving visual (spectrograms) and auditory (audio) elements, with creative tasks like naming sounds to spark interest.
  4. Merging citizen science with entertainment-focused design to explore methods for enhancing perception and memory.

• Implementation Steps: The challenge activity is structured in multiple levels, including:

  1. Guidance: Bob Bristlebird guides users in interpreting audio information.
  2. Matching Tasks: Users match natural soundscape fragments with bird audio, progressing from simple to complex tasks.
  3. Free Exploration: Users freely discover bird audio features and learn about other species.
  4. Naming Tasks: Users creatively name bird calls based on their shapes or sounds.
  5. Task Application: Users apply their knowledge to identify Eastern Bristlebird calls in real soundscape audio.

Research Outcomes

• Specific Outcomes:

  1. Effectiveness of Emotional and Guiding Support: Bob Bristlebird, as a conversational agent, successfully provided users with emotional comfort and learning support.
  2. Integration of Tasks and Learning: Various gamified tasks helped users gradually master complex cross-modal perception (visual and auditory).
  3. Creative Naming Enhances Memory: Personal naming and interpretation of natural sounds increased sound recognizability and memory retention.
  4. Stimulating Ecological Interest: Users expressed willingness to learn more about the Eastern Bristlebird and its survival conditions.

• Advantages: Compared to existing solutions, the Bristle Whistle Challenge leverages multisensory and gamified design to enhance user interest and learning capabilities, introducing new ways to engage with environmental audio and citizen science.

• Experimental and Evaluation Results:

  1. Game testing showed that users found the experience enjoyable and actively experimented with different audio identification strategies.
  2. Users adopted various self-directed learning methods, including musical terminology metaphors, sound imitation, and visual associations.
  3. Creative naming of bird calls significantly improved memory and identification abilities.

• Limitations and Future Directions:

  1. Although gamified design proved effective, the diversity of participants and backgrounds requires further validation of its universality.
  2. Lack of quantifiable assessment of long-term learning and skill improvement.
  3. Future work could explore extending the outcomes to diverse audiences and fostering long-term collaboration within citizen science initiatives.