Roomify: Spatially-Grounded Style Transformation for Immersive Virtual Environments

Paper Title

Roomify: Spatially-Grounded Style Transformation for Immersive Virtual Environments

Publication Info

• Topic area: Virtual reality systems integrating physical and virtual environments.
• Keywords: Spatial awareness, virtual reality, mixed reality, immersive environments, AI-driven generation, scene transformation, user-centered design, creative prototyping, functional semantics, geometric preservation.

Background and Problem

• Problem / challenge: Current VR systems struggle to balance immersive experiences with spatial awareness, often sacrificing one for the other. Passthrough solutions break immersion, while fully virtual environments compromise spatial awareness and safety.
• Significance: Spatial awareness is critical for safety and user experience in VR, especially in home environments and professional design workflows. Bridging physical and virtual spaces can enhance immersion and creative applications.
• Motivation and related work: Existing approaches like boundary systems, passthrough methods, and scene proxies fail to integrate physical and virtual environments seamlessly. They either disrupt immersion or lack flexibility in transformation. Roomify builds on prior work in spatial understanding, generative stylization, and AI-assisted authoring to address these gaps.

Solution

• Proposed approach: Roomify—a spatially-grounded transformation system that generates themed virtual environments anchored to users’ physical rooms while preserving spatial structure and functional semantics.
• Novelty:
  1. A spatially-grounded generation pipeline balancing stylistic flexibility with geometric and functional preservation.
  2. A cross-reality authoring tool combining MR for spatial grounding and VR for immersive preview.
  3. Empirical validation demonstrating improved presence, spatial awareness, and creative expressiveness compared to existing methods.
• Procedure and key techniques:
  • Scene Understanding: Monocular RGB video is processed using SLAM3R and SpatialLM to reconstruct room geometry and parse spatial semantics.
  • Style Extraction and Mapping: Multimodal inputs generate structured style specifications, ensuring functional consistency, style coherence, and spatial alignment.
  • Content Generation: Stylized assets are created using reference-guided workflows for objects, boundary elements, and skyboxes.
  • Scene Composition: Generated assets are registered into cohesive virtual environments, preserving spatial relationships and functional roles.

Results

• Concrete findings:
  • Roomify improves presence by 63% over passthrough and 26% over fully virtual baselines.
  • Spatial awareness is significantly better than fully virtual environments.
  • Scene quality rated at 5.95/7 and creativity support at 6.08/7 by design professionals.
• Advantage over baselines:
  • Combines immersive transformation with spatial awareness, outperforming passthrough and fully virtual conditions in presence and user experience.
  • Superior scene quality and creativity support compared to AI re-texturing and text-to-3D generation methods.
• Experiments / evaluation:
  • Study 1: 18 VR users evaluated Roomify against passthrough and fully virtual baselines in entertainment and navigation tasks.
  • Study 2: 8 design professionals compared Roomify with AI re-texturing and text-to-3D generation for creative prototyping.
  • Metrics included presence, spatial awareness, scene quality, creativity support, and navigation performance.
• Limitations and future work:
  • Static environment assumption; does not handle dynamic interactions or real-time spatial tracking.
  • Limited fine-grained control over generated content.
  • Evaluation focused on individual experiences; multi-user applications remain unexplored.

Summary

Roomify introduces a spatially-grounded transformation system that integrates physical room geometry into immersive virtual environments, balancing stylistic diversity with spatial and functional consistency. Empirical studies demonstrate significant improvements in presence, spatial awareness, and creative expressiveness compared to existing VR approaches. By enabling users to transform their physical spaces into personalized virtual environments, Roomify supports applications ranging from themed entertainment to professional design prototyping. Future work will address dynamic environments, multi-user scenarios, and enhanced control mechanisms.