Draped Surfaces: A Contour-Adaptive Interface Overlaid on the Physical Environment for Mixed Reality Workspaces

Paper Title

Draped Surfaces: A Contour-Adaptive Interface Overlaid on the Physical Environment for Mixed Reality Workspaces

Publication Info

• Topic area: Mixed Reality (MR) workspace design and interaction
• Keywords: Mixed Reality, contour-adaptive interface, workspace design, legibility, environmental understanding, user interaction, deformable interfaces, spatial integration, augmented reality, interface usability

Background and Problem

• Problem / challenge: Conventional MR workspaces often rely on flat, mid-air interfaces that obstruct users' view of their physical environment, limiting interaction with real-world objects. Existing solutions, such as semi-transparent interfaces or projection-based systems, fail to fully address the trade-off between digital content legibility and environmental awareness, especially on complex geometries.
• Significance: Addressing this challenge is crucial for creating MR workspaces that support both immersive digital interaction and seamless integration with the physical world, enabling more natural and efficient user experiences.
• Motivation and related work: Prior work has explored workspace expansion, legibility on complex surfaces, and environmental understanding in MR interfaces. However, these approaches often fail to balance legibility and spatial awareness, particularly when adapting to irregular physical geometries. This paper builds on these efforts by introducing a deformable interface that adapts to real-world contours.

Solution

• Proposed approach: Contour-Adaptive Mixed Environment Overlays (CAMEO), a deformable MR interface that drapes virtual windows over physical surfaces, balancing digital content legibility with environmental understanding.
• Novelty:
  1. Introduction of a contour-adaptive workspace that integrates digital content with real-world geometry.
  2. Empirical evidence demonstrating reduced hand-movement detours and preserved text legibility on complex surfaces.
  3. Exploration of new design opportunities for deformable MR workspaces, including seamless UI flow and semantic positioning.
• Procedure and key techniques:
  • CAMEO uses a two-stage process: initial shape formation (based on user viewpoint and physical surface contours) and surface smoothing (via Laplacian-based diffusion).
  • Key metrics, such as View Efficiency and Space Efficiency, are used to balance legibility and spatial integration.
  • The system adapts interface depth and smoothing iterations to optimize usability while preserving environmental context.

Results

• Concrete findings:
  • Draped interfaces reduce background occlusion and increase available foreground volume compared to flat interfaces.
  • CAMEO reduces hand-movement detours by 19.79% in light environments and 9.43% in heavy environments.
  • Text legibility on draped surfaces remains comparable to flat interfaces, with no significant differences in reading performance.
  • Draped interfaces improve users' ability to infer the shapes of objects behind the interface, achieving a balanced trade-off between shape awareness and legibility.
• Advantage over baselines:
  • CAMEO outperforms flat MR interfaces in reducing interaction inefficiencies and maintaining environmental understanding.
  • Draped configurations offer complementary benefits to transparency, enhancing usability and user preference.
• Experiments / evaluation:
  • Simulation studies evaluated occlusion, foreground volume, and interface visibility across viewpoints.
  • User Study I assessed hand movement and subjective usability in interaction tasks.
  • User Study II examined text legibility and shape awareness across different draping conditions.
  • Participants included 16 and 24 users in User Studies I and II, respectively, with tasks conducted in controlled MR environments.
• Limitations and future work:
  • The method assumes accurate physical object meshes, which may require further integration with scanning technologies.
  • Results may not generalize to all interface sizes, shapes, or real-world contexts.
  • Future work could explore dynamic or multi-user scenarios, broader interface designs, and computational approaches to optimize legibility.

Summary

This paper introduces CAMEO, a contour-adaptive MR interface that drapes virtual windows over physical surfaces to balance digital content legibility and environmental understanding. Empirical studies demonstrate that CAMEO reduces interaction inefficiencies, preserves text legibility, and supports shape awareness, outperforming conventional flat MR interfaces. The approach opens up new design opportunities for seamless UI flow and semantic positioning, enabling more natural integration of virtual workspaces with real-world contexts. Future research will focus on expanding CAMEO's applicability to diverse environments and exploring advanced interface designs.