Beyond Links: Exploring Visual Representations of Multi-View Relations in Mixed Reality

Paper Title

Beyond Links: Exploring Visual Representations of Multi-View Relations in Mixed Reality

Publication Info

• Topic area: Visual representation of multi-view relations in Mixed Reality environments.
• Keywords: Mixed Reality, multi-view systems, visual associations, spatial layouts, immersive environments, design space, interaction design, visualization techniques, Gestalt principles, prototyping framework.

Background and Problem

• Problem / challenge: Mixed Reality (MR) environments lack systematic guidance for representing relations between multiple views, unlike 2D desktop environments where extensive recommendations exist.
• Significance: Addressing this gap is essential for improving exploration, coordination, and sensemaking in MR applications, which are increasingly used for complex information tasks.
• Motivation and related work: While prior research on 2D multi-view systems has established guidelines for linking, coordinating, and arranging views, MR introduces spatial affordances and challenges (e.g., occlusion, field of view limitations). Existing MR studies often adapt 2D paradigms but fail to systematically address how spatial, perceptual, and interaction constraints impact view relations.

Solution

• Proposed approach: Development of a design space for visual association techniques tailored to immersive multi-view environments, supported by a lightweight prototyping framework and envisioning scenarios.
• Novelty:
  1. Identification and description of seven association strategies and patterns for representing view relations in MR.
  2. Introduction of a design space comprising three components (view, relation, association) and 27 dimensions.
  3. Implementation of a lightweight prototyping framework to explore and refine association techniques.
• Procedure and key techniques:
  • Systematic literature review to construct and refine a codebook of association techniques.
  • Iterative coding and validation of 44 immersive multi-view papers.
  • Synthesis of recurring strategies into a structured design space.
  • Development of a Unity-based prototyping tool for MR applications.
  • Application of the design space to three use cases: traffic monitoring, storytelling, and sensemaking.

Results

• Concrete findings:
  • Seven association strategies identified: Link, Planar and Cylindrical Uniform Grid, Slideshow with Thumbnails, Cluster and Containment, Stack and Ripple, Temporal Reading Order, Reference and Anchor.
  • Design space includes 27 dimensions across three components: view (e.g., modality, dimensionality), relation (e.g., type, cardinality), and association (e.g., spatial layout, visual marks, visual effects).
• Advantage over baselines:
  • Provides structured guidance for MR-specific association design, addressing spatial, perceptual, and interaction constraints unique to immersive environments.
  • Bridges the gap between 2D visualization research and MR applications.
• Experiments / evaluation:
  • Demonstrated the design space through three envisioning scenarios:
    1. Traffic monitoring with layered spatial partitions and anchored vehicle camera feeds.
    2. Storytelling with clustered evidence views and dynamic visual effects for narrative progression.
    3. Sensemaking with thematic maps anchored to physical landmarks and mediated links for provenance.
• Limitations and future work:
  • Limited exploration of interaction techniques for associations.
  • Need for controlled user studies to validate Gestalt principles and visual mark effectiveness in MR.
  • Opportunities for expanding MR-native association strategies, multimodal cues, and hybrid setups involving physical and virtual displays.

Summary

This paper addresses the lack of systematic guidance for representing multi-view relations in Mixed Reality environments by introducing a design space comprising three components and 27 dimensions. Seven association strategies were identified and validated through literature review and envisioning scenarios, demonstrating their applicability in traffic monitoring, storytelling, and sensemaking tasks. A lightweight prototyping framework was developed to enable iterative design and refinement of associations. The contributions bridge 2D visualization research with MR applications, providing structured guidance for designing legible and expressive multi-view environments. Future work includes exploring interaction techniques, multimodal cues, and hybrid setups involving physical displays.