GastroConcerto: Towards Designing Dining–Sound Pairings to Support Culinary Creativity

Paper Title

GastroConcerto: Towards Designing Dining–Sound Pairings to Support Culinary Creativity

Publication Info

• Topic area: Auditory interaction design in Human-Food Interaction (HFI) to enhance culinary creativity.
• Keywords: auditory interaction, culinary creativity, dining–sound pairing, Human-Food Interaction, sonic dishes, chefs, mindful dining, food expressiveness, interactive systems, digital gastronomy.

Background and Problem

• Problem / challenge: Existing auditory interaction systems in HFI are often limited to lab-controlled settings, rely on predefined sound outputs, and fail to integrate with chefs' daily practices. These systems do not allow chefs to fully express their culinary creativity through meaningful auditory interactions.
• Significance: Enabling chefs to design auditory interactions can deepen diners' engagement, enhance sensory experiences, and transform food into a multisensory medium for creative expression.
• Motivation and related work: Prior research has explored sound's influence on dining experiences and developed edible auditory interfaces and interactive systems. However, these approaches often overlook the interactive space between diners and food, limiting chefs' ability to craft personalized and expressive auditory interactions.

Solution

• Proposed approach: GastroConcerto, an auditory dining system that enables chefs to design dining–sound pairings, where specific sounds are triggered by diners' interactions with food.
• Novelty:
  1. Introduces dining–sound pairing as a new interactive mode for culinary creativity.
  2. Shifts the design of auditory interactions from interface designers to chefs.
  3. Provides a system that integrates into chefs' daily practices, supporting iterative creation and mindful dining experiences.
  4. Demonstrates the use of food properties and dining interactions to create expressive sonic dishes.
• Procedure and key techniques:
  • Developed a system with a magnetic contact microphone, a dining platform, and a companion app.
  • Conducted a formative study to identify design considerations, including hygiene, usability, and sound expertise.
  • Evaluated the system through a within-subjects study and a five-day field deployment.
  • Used machine learning (Random Forest) to classify dining interactions and assign sounds.

Results

• Concrete findings:
  • Dining–sound pairing mode significantly enhanced chefs' culinary creativity, with improvements in enjoyment (p = .010), exploration (p = .015), expression (p = .026), and perceived worth of effort (p = .010).
  • The system achieved a 93.10% accuracy in matching auditory responses to dining interactions.
  • Behavioral metrics showed increased engagement, including longer creation durations (p < .001) and more frequent interactions (p = .003).
• Advantage over baselines:
  • Compared to sequential sound playback, dining–sound pairing fostered deeper engagement, more meaningful interactions, and greater sensory and emotional connection with dishes.
• Experiments / evaluation:
  • Study 1: Within-subjects experiment with 16 chefs comparing dining–sound pairing to sequential playback.
  • Study 2: Five-day field deployment with six chefs, culminating in a dining session with invited diners.
  • Metrics included Creativity Support Index (CSI), dining awareness, sensory appeal, and emotional engagement.
• Limitations and future work:
  • Limited to tableware-based interactions, excluding cultural practices like eating by hand.
  • Time-consuming setup for chefs; future systems could include pre-trained action profiles.
  • Small-scale studies in controlled settings; future work should explore deployment in diverse real-world environments.

Summary

GastroConcerto introduces a novel auditory dining system that empowers chefs to design dining–sound pairings, enhancing culinary creativity and fostering mindful dining experiences. The system demonstrated significant improvements in chefs' creative engagement and diners' sensory awareness. By leveraging food properties and dining interactions, GastroConcerto transforms food into a multisensory medium for expression. Future research could explore modular components, automated classification, and broader deployment to further integrate auditory interaction into diverse culinary contexts.