Nudging the Somas: Exploring How Live-Configurable Mixed Reality Objects Shape Open-Ended Intercorporeal Movements

Paper Title

Nudging the Somas: Exploring How Live-Configurable Mixed Reality Objects Shape Open-Ended Intercorporeal Movements

Publication Info

• Topic area: Mixed Reality (MR) systems for collective movement and interaction.
• Keywords: Mixed Reality, intercorporeal movement, nudging, live configuration, embodied interaction, digital physics, synesthetic feedback, agency, improvisation, soma design.

Background and Problem

• Problem / challenge: Prior MR systems often focus on individual, goal-oriented movement guidance or direct actuation, limiting exploration and reducing agency. The potential of MR objects to shape collective, co-located movement in open-ended, improvisational contexts remains underexplored.
• Significance: Understanding how MR objects can guide group dynamics without scripting actions has implications for creative practices, social play, and interaction design, fostering emergent coordination while preserving agency.
• Motivation and related work: Previous work on MR nudging has demonstrated its ability to influence individual trajectories but has not addressed collective movement. Research in movement computing, soma design, and improvisational dance highlights the importance of embodied, relational dynamics, which this paper seeks to extend into MR systems.

Solution

• Proposed approach: GravField, a co-located MR performance system where an Object Jockey (OJ) live-configures virtual objects (e.g., ropes, springs, magnetic fields) with parameterized digital physics to nudge participants’ movements through synesthetic audiovisual feedback.
• Novelty:
  1. Introduction of live-configurable MR objects (MROs) to nudge collective movement while preserving open-endedness.
  2. Empirical analysis of emergent group coordination and negotiation facilitated by MROs.
  3. Development of the concept of MR Live Nudges and the MR-mediated intercorporeal perception–action loop.
  4. Derivation of six actionable design strategies for non-prescriptive nudging in MR systems.
• Procedure and key techniques:
  • Iterative development of GravField through four stages: single-player sound-only sketching, multi-player sound-only exploration, addition of MR visuals, and live configuration by OJs.
  • Integration of synesthetic audiovisual mappings to simulate digital physics and enhance perceptual coupling.
  • Performance-led workshops with 25 participants to evaluate the system, using video analysis, interviews, and soma trajectory sketches.

Results

• Concrete findings:
  • Participants progressed through a six-stage Live Nudging Spiral: learning, internalizing, coordination, exploration, boredom, and reconfiguration.
  • MROs shaped distinct movement patterns: Rope encouraged mirrored swinging, Spring prompted tension-and-release dynamics, and Magnetic Field elicited exploratory navigation.
  • Synesthetic feedback loops reinforced action, enabling participants to perceive and respond to digital physics intuitively.
• Advantage over baselines:
  • GravField supports emergent coordination and creative exploration without scripting actions, contrasting with prescriptive motion guidance or actuation-based systems that limit agency.
• Experiments / evaluation:
  • Workshops with 25 participants (backgrounds in movement, music, and interactive art) conducted in a semi-public setting.
  • Data collected included video recordings, audio, post-session interviews, group discussions, and soma trajectory sketches.
  • Analysis focused on intercorporeal coordination, participant engagement, and the role of OJ interventions.
• Limitations and future work:
  • Limited generalizability due to reliance on two skilled OJs; future work should train a broader pool of facilitators and develop transferable practices.
  • Narrow sensing scope (head movement only); expanding to whole-body sensing could enhance interaction diversity.
  • Restricted MRO repertoire; additional metaphors could broaden applicability across contexts.
  • Ethical considerations for MR nudging in public spaces require further exploration.

Summary

GravField introduces a novel approach to shaping collective movement in co-located MR environments through live-configurable virtual objects that leverage intuitive metaphors and synesthetic audiovisual feedback. Empirical findings reveal that these objects foster emergent coordination, sustain exploratory engagement, and support fluid negotiation of agency among participants and facilitators. The study identifies a recurrent Live Nudging Spiral and proposes actionable design strategies for MR systems that preserve open-endedness while guiding group dynamics. GravField’s insights contribute to the development of programmable realities, offering new possibilities for creative practices and embodied interaction design.