OpenMic: Utilizing Proxemic Metaphors for Conversational Floor Transitions in Multiparty Video Meetings

Title of the Paper

OpenMic: Utilizing Proxemic Metaphors for Conversational Floor Transitions in Multiparty Video Meetings

Bibliographic Information

• Subject Area: Human-Computer Interaction (HCI) / Video Conferencing System Design
• Keywords: Multiparty video conferencing, video meetings, turn-taking, nonverbal communication, proxemics

Research Background and Problem

• Identified Problems or Challenges:

  • Turn-taking is a major interaction challenge in multiparty remote meetings, especially in the remote work environment during the COVID-19 pandemic. Users often struggle with overlapping and interruptive conversations in video meetings.
  • Current meeting tools lack visual designs that support social and spatial cues (proxemic cues), which are critical in human communication.
  • Existing research on video conferencing and social virtual reality is limited by the need for specialized equipment or focuses only on entry and exit dynamics, without adequately supporting turn-taking dynamics.

• Significance of the Research:

  • Supporting turn-taking dynamics can not only improve interaction but also alleviate common audio and visual management issues.
  • Proxemic theory (theory of spatial distance) can serve as a novel approach to designing video conferencing tools, helping users understand intentions more naturally.

• Research Motivation and Related Work:

  • Previous studies have developed various video conferencing tools based on spatial cues, such as 2D or 3D virtual spaces, but these systems lack designs tailored for turn-taking dynamics.
  • In multiparty meetings, turn-taking is achieved through verbal communication and nonverbal cues, forming structures for collaboration and problem-solving. These dynamic structures require new digital interaction methods for support.

Solution

• Proposed Method or Solution:

  • A prototype video conferencing system (OpenMic) was developed, utilizing spatial metaphors to support turn-taking dynamics. The solution includes:
    1. Virtual Floor: A fixed-feature spatial interface for conveying speaking intentions and managing microphone control.
    2. Malleable Mirrors: Users can dynamically adjust the size and position of video windows to express their proximity and willingness to participate in the conversation.

• Innovations:

  • OpenMic introduces spatial metaphors into video conferencing for the first time to support nonverbal management of turn-taking.
  • It provides continuous, gradual visual transitions in a multiparty environment, as opposed to traditional binary switches (e.g., raise-hand buttons, mute/unmute).

• Implementation Steps and Technology:

  • A 2D interface was constructed, allowing users to drag video windows into the "virtual floor."
  • WebSockets were used to manage data transmission between the front-end and back-end, supporting more user video interactions.
  • By adjusting the relative size and position of video windows, users can actively express their intent to participate in the conversation.

Research Outcomes

• Specific Findings:

  • User studies revealed that OpenMic's two main features (Virtual Floor and Malleable Mirrors) improved the flow of turn-taking in multiparty video meetings.
  • The study analyzed turn-taking dynamics under spatial metaphors, including how participants used video window adjustments to position themselves in the conversation and influence interactions.

• Advantages Compared to Existing Solutions:

  • Through dynamic interaction with video windows, OpenMic provides more nuanced nonverbal interaction support than traditional video conferencing tools.
  • It enhances users' visual awareness of "speaking intentions" and "participation zones," reducing conversational chaos.

• Experimental or Evaluation Results:

  • Experiments showed significant differences in turn-taking behavior across different group sizes (small groups of 4, medium groups of 8).
  • Speaking zones on the virtual floor were divided into functional areas (e.g., central zone, transition zone, peripheral zone, and audience zone), supporting gradual transitions into speaking states.

• Limitations and Future Directions:

  • Group size scalability: Currently, OpenMic is suitable for small and medium-sized meetings. Future work should explore designs that support larger groups.
  • Video window operation burden: Manually adjusting windows may increase user interaction load. Future research could investigate automation-based clustering and layout optimization.
  • Coordination challenges for moderators: Communication among multiple speakers on the virtual floor may lead to management conflicts. Future designs should include additional channels to support moderator collaboration.

Conclusion

This paper proposes a novel video conferencing system design based on proxemic theory, introducing virtual floors and dynamically adjustable video windows to provide a new approach to managing turn-taking. Exploratory studies demonstrated significant results, indicating that the design can help facilitate speaking transitions through visual cues, reduce conversational chaos, and enhance user collaboration experiences. Future work should expand the system to support larger user groups and incorporate automated transition designs.