Engaging Passers-by with Rhythm: Applying Feedforward Learning to a Xylophonic Media Architecture Facade

Title of the Paper

"Engaging Passers-by with Rhythm: Applying Feedforward Learning to a Xylophonic Media Architecture Facade"

Paper Information

• Subject Area: Human-Computer Interaction (HCI), Media Architecture, Public Space Design
• Keywords: Interactive Architecture, Media Architecture, Human-Building Interaction, Public Displays, Placemaking, Rhythm, Musical Interface, Musical Expression, Feedforward Learning, Comparative Study, Field Study

Research Background and Problem

• Problem Identification:

  • Traditional media architecture primarily relies on visual stimuli; this study explores the potential of attracting passers-by through sound and rhythm.
  • Passers-by often fail to realize that interactive installations are designed based on their movements or do not understand how to control the rhythm.
  • A key challenge is how to encourage passers-by to not only linger longer but also purposefully control and engage with the rhythm.

• Significance:

  • Media architecture can enhance people's understanding and engagement with public spaces, fostering community interaction and a sense of place.
  • Exploring non-visual interaction forms, such as tactile and auditory, contributes to diversifying media architecture design.

• Research Motivation:

  • To address the limitations of existing rhythm-based interaction designs in attracting passers-by, the study introduces the concept of "feedforward learning."
  • Feedforward learning enhances user awareness and engagement by informing them of potential outcomes before they act.

• Related Work:

  • Rhythm-based interactive designs have significant effects on walking behaviors, such as gait control and staircase therapy.
  • "Feedforward" has been applied in gesture interfaces, autonomous driving, and self-modeling, proving its effectiveness in enhancing learning and guidance.

Solution

• Method or Solution:

  • Developed an interactive architectural facade called MUURmelaar, which interacts with passers-by through sound and rhythm.
  • Utilized feedforward learning to drive the architectural rhythm, guiding passers-by to engage and enhancing the persistence and intentionality of the interaction experience.

• Innovations:

  • Applied "feedforward learning" to the abstract design of architectural interaction.
  • Combined camera tracking with architectural rhythm responses to create an interaction between rhythm and real-time passer-by behavior.

• Implementation Steps:

  1. Stage Design: Divided the interaction into three stages—Passive, Active, and Exploratory.
     • Passive: Arouse curiosity through basic rhythms and textual instructions.
     • Active: Guide passers-by to learn interaction with the architecture through feedforward rhythms.
     • Exploratory: Allow passers-by to freely improvise and control through feedback rhythms.
  2. Hardware and Software Implementation:
     • Used cameras to capture passer-by positions and movements.
     • Transformed the data into rhythmic responses on the architectural facade using custom algorithms.
  3. Experiment Design:
     • Comparative Study: Invited 25 participants to experience feedforward and feedback-driven conditions.
     • Field Study: Operated the installation in a natural environment for a month to observe passer-by interaction behaviors.

Research Outcomes

• Specific Findings:

  • The feedforward-driven condition significantly enhanced passer-by engagement time and intentionality.
  • Textual guidance and rhythmic rewards effectively motivated passers-by to explore interactive behaviors.
  • Identified two metrics influencing interaction quality:
    1. Reading Time: The time spent reading textual instructions positively correlated with interaction quality.
    2. Learning Rate: The speed of unlocking the first two rhythms predicted the depth of subsequent interactions.

• Advantages Over Existing Solutions:

  • Significantly prolonged passer-by interaction duration and depth.
  • Introduced new possibilities for non-visual interaction modes.
  • The combination of text and rhythm was critical in attracting passers-by.

• Experimental or Evaluation Results:

  • In the comparative study, the feedforward condition received higher user satisfaction scores (SUS scores).
  • In the field study, passer-by interaction duration significantly increased compared to previous years.

• Limitations and Future Directions:

  • Did not fully explore the potential of rhythm to create melodies.
  • Did not cover long time spans, leaving the long-term performance of media architecture unexplored.
  • Cultural and seasonal factors may have influenced the results, requiring broader contextual validation.
  • Future work could optimize rhythm design to better align with the social and environmental needs of the architecture.

Design Recommendations

• Use feedforward learning to guide newcomers into interaction while providing shortcuts for experienced users to skip the learning phase.
• Dynamically adjust interaction difficulty based on the time spent reading textual instructions.
• Utilize learning rate metrics to optimize subsequent interaction experiences, enhancing passer-by engagement and intentionality.
• Root interactive rhythms in spatial and social contexts, using sound and rhythm to make architecture more attuned to the surrounding environment and community needs.

The structured presentation above highlights the study's logic and contributions while providing valuable insights for future research in related fields.