MirrorBlender: Supporting Hybrid Meetings with a Malleable Video-Conferencing System

Title of the Paper

MirrorBlender: Supporting Hybrid Meetings with a Malleable Video-Conferencing System

Paper Information

• Research Domain: Human-Computer Interaction, Video Conferencing Technology, Hybrid Meetings
• Keywords: Hybrid Meetings, Video Conferencing, Hybrid Interaction Space, Adjustable Dynamic Media, Inclusive Meetings, Nonverbal Communication, Collective Decision Support

Research Background and Problem

• Problem and Challenges: In hybrid meetings, the lack of spatial consistency makes it difficult for remote participants to attract attention or effectively engage through nonverbal cues. This phenomenon, dominated by the main meeting room, often leaves remote participants feeling excluded.
• Significance: With remote work becoming increasingly prevalent, optimizing the interactive experience of hybrid meetings is crucial, particularly in enhancing the sense of presence and engagement for remote participants.
• Research Motivation: Current video conferencing technologies are often limited to providing low-cost, stable solutions but lack sufficient support for nonverbal interactions, especially deictic gestures. Addressing this issue can improve inclusivity and efficiency in hybrid meetings.
• Related Work: Previous studies have attempted to support nonverbal communication using physical and virtual frameworks (e.g., ClearBoard and HyperMirror), but these approaches are expensive and inflexible. This has motivated researchers to explore more flexible and cost-effective software-based solutions.

Solution

• Method and Innovation:

  • Proposed a novel adjustable hybrid video conferencing system—MirrorBlender.
  • MirrorBlender provides a shared virtual 2D space (WYSIWIS, What-You-See-Is-What-I-See), allowing users to dynamically adjust and blend mirrored video streams (including camera and screen mirrors).
  • Introduced continuous interaction features for mirror operations (zooming, panning, transparency adjustments) to support inclusivity and nonverbal communication.
  • Enabled mirror blending functionality, allowing users to overlay video streams to achieve "fusion" of virtual and physical spaces.

• Key Technologies and Implementation Steps:

  1. Developed a foundational WYSIWIS shared framework to ensure all users see the same 2D layout synchronously.
  2. Implemented mirror operation features, including adjustments to position, size, and transparency, while enabling overlapping mirrors.
  3. Achieved integration of non-physical spaces through dynamic overlapping and transparency adjustments, creating optical illusions.
  4. Built a cross-platform application using WebRTC and screen-sharing APIs, operable in standard web browsers.

• Research Methodology:

  • Deployed MirrorBlender in hybrid meetings involving three groups of five participants each, conducting task-driven system tests.
  • Collected preliminary qualitative data to uncover the system's role in supporting nonverbal interactions and participant behavior dynamics.

Research Findings

Specific Outcomes

1. Innovative Opportunities Supported by the System:

   • Optical Illusions and Interactive Playfulness: Users created gamified optical illusions through mirror adjustments (e.g., combining remote participants' avatars with physical objects or co-located participants' bodies).
   • Support for Diverse Deictic Gestures: Users expressed spatial references through body gestures using mirrors, such as dividing work areas with both hands or pointing to screen content via mirrors.
   • Dynamic Attention Regulation: Users guided group attention by adjusting mirror attributes, ranging from subtle changes (e.g., moving mirrors) to prominent actions (e.g., enlarging mirrors and increasing opacity).

2. Improved Equality and Inclusivity:

   • Mirror adjustments enabled remote participants to actively integrate into meeting activities, enhancing their perception of the main meeting room and fostering smoother interactions.
   • Disrupted traditional power dynamics in hybrid meetings that often hinder equal participation for remote attendees.

3. Enhanced Support for Nonverbal Communication:

   • Bridged physical and virtual frameworks, allowing users to customize collaborative layouts to accommodate diverse collaboration styles and nonverbal interactions.

Comparison with Existing Solutions

• Advantages: MirrorBlender reduces dependency on expensive hardware or fixed spatial layouts, offering high flexibility and adjustability to support diverse scenarios. It surpasses existing video conferencing systems in interface interactivity and user control.
• Disadvantages: Current mirror-based pointing remains limited by parallax effects, especially among co-located participants.

Experimental and Evaluation Results

• Participant Feedback: Most participants reported new interaction opportunities and a sense of inclusivity brought by remote mirroring, although occasional cognitive effort was required for gesture calibration.
• Dynamic Adaptability: Different groups demonstrated various collaboration styles supported by the system's flexibility, ranging from clearly divided task allocation to highly coupled collective decision-making.

Limitations and Future Directions

1. Applicability and Scalability: Further research is needed to assess the system's applicability for large-scale meetings or real organizational environments.
2. Technical Optimization:
   • Address parallax issues (particularly in multi-screen physical setups).
   • Introduce intelligent preset layouts or background removal to reduce visual clutter.
3. User Training: Explore more natural user learning curves to minimize cognitive load.
4. Task Diversity: Evaluate the system in more real-world application scenarios, such as education, telemedicine, or team creative sessions.

Conclusion

MirrorBlender offers a novel and effective design for hybrid meeting systems, significantly enhancing nonverbal communication capabilities and inclusivity for remote participants. It represents a critical step toward achieving "beyond presence" hybrid meeting experiences while revealing further research opportunities in multi-screen collaboration design.