Through Their Eyes and In Their Shoes: Providing Group Awareness During Collaboration Across Virtual Reality and Desktop Platforms

Title of the Paper

Through "Their Eyes" and "Their Shoes": Enabling Group Awareness for Collaboration Across Virtual Reality and Desktop Platforms

Bibliographic Information

• Subject Area: Human-Computer Interaction and Collaborative Data Visualization
• Keywords: Asymmetric Collaboration, Virtual Reality, Immersive Analytics, Ubiquitous Analytics, Cross-Virtuality

Research Background and Problem Statement

• Problems and Challenges:

  • Issues in collaboration between users of different devices: Differences in physical devices, display modes, and interaction models between desktop users and virtual reality (VR) users reduce "common ground," complicating collaboration. For instance, desktop users and VR users may have significantly different understandings of the same data.
  • Particularly in collaboration across the Reality–Virtuality Continuum (RVC), there is often a lack of "group awareness" capabilities (i.e., the ability to understand the activities of other team members) between desktop and head-mounted VR devices. For example, users may not be able to see each other's operations or viewpoint states.

• Significance:

  • Cross-device collaboration without group awareness reduces work efficiency and increases misunderstandings or interaction delays.
  • With the rise of Immersive Analytics, VR and Mixed Reality (MR) are increasingly used to support complex data analysis tasks, necessitating more universal and effective collaborative solutions.

• Motivation and Related Work:

  • Previous research has explored the roles of collaborative visualization in synchronous and asynchronous, co-located and remote environments, but there is insufficient research on how to maintain group awareness in cross-device collaboration, especially in cross-virtuality analysis systems.
  • Although some studies have proposed specific group awareness mechanisms (e.g., remote touch, cursor tracking), there is a lack of a unified design philosophy and comprehensive research evaluation.

Proposed Solution

• Proposed Method or Solution:

  • The study proposes the "Eyes-and-Shoes" design principles to support group awareness across heterogeneous devices. These principles include four levels, ranging from the lowest to the highest:
    1. Landmarks and Analogous Views (L1): Paired visual metaphors and interactions to make references more intuitive.
    2. Information Cues (L2): For example, shared cursor positions, eye-tracking focus, and other cues to emphasize members' work content.
    3. Interaction Sharing (L3): Synchronized interactions and visualization states, allowing users to share operational processes across devices.
    4. Viewpoint Sharing (L4): Enabling higher-level collaboration by viewing collaborators' real-time perspectives and operations through screen sharing and similar methods.

• Innovations:

  • The study introduces a unified "Eyes-and-Shoes" model that integrates input (Interaction) and output (View) as the core of collaborative group awareness.
  • These principles are concretized within the framework of cross-virtuality collaboration and are systematically validated for the first time in both immersive and non-immersive environments.

• Implementation Steps and Key Technologies:

  • Developed the experimental platform VRxD, which supports synchronous collaborative analysis between desktop and VR, enabling real-time data synchronization, interaction state sharing, and video stream transmission.
  • Examined two scientific scenarios: abstract data visualization (e.g., parallel coordinate plots) and natural spatial mapping visualization (e.g., real-world baseball trajectory data), conducting interaction experiments for each.

Research Findings

• Results and Advantages:

  1. Validation of Design Principles:
     • The "Eyes-and-Shoes" principles were proven effective, as participants achieved high levels of group awareness using this approach.
     • Higher levels of group awareness support (L3 or L4) are required when there are significant differences between devices and views in collaboration.
  2. User Study Findings:
     • L1 and L2 techniques suffice for quick referencing and simple tasks, while L3 and L4 are more suitable for high-demand collaborations, such as building or rebuilding group awareness in complex tasks.
     • Although L4 (viewpoint sharing) is more comprehensive, L3 (interaction sharing) is deemed more suitable for short-term task requirements.
     • Different view types (e.g., abstract vs. natural spatial mapping) significantly influence collaborative behavior. Abstract tasks rely more on L3, while natural tasks rely more on L1.

• Experimental and Evaluation Results:

  • In tasks involving abstract and natural spatial mapping of baseball data, participants exhibited different interaction and collaboration patterns when solving complex problems.
  • The viewpoint sharing feature enabled VR users to explore data more effectively, while desktop users utilized interaction sharing for data input and precise positioning.

• Limitations and Future Directions:

  • Limitations:
    • The design principles and data visualization schemes in the experiments may not be universally applicable across all domains, requiring domain-specific testing.
    • Individual differences (e.g., proficiency with devices, asymmetric domain expertise) may influence experimental results.
  • Future Design:
    • Enhance the information cue level (L2), such as by adopting stronger viewpoint "annotation" features.
    • Investigate new viewpoint sharing technologies (e.g., tighter integration of multi-device 3D-2D mapping systems).
    • Conduct long-term usability testing in real-world domain tasks (e.g., medical or engineering design).