"In VR, everything is possible!": Sketching and Simulating Spatially-Aware Interactive Spaces in Virtual Reality
Authors
We propose using virtual reality (VR) as a design tool for sketching and simulating spatially-aware interactive spaces. Using VR, designers can quickly experience their envisioned spaces and interactions by simulating technologies such as motion tracking, multiple networked devices, or unusual form factors such as spherical touchscreens or bezel-less display tiles. Design ideas can be rapidly iterated without restrictions by the number, size, or shape and availability of devices or sensors in the lab. To understand the potentials and challenges of designing in VR, we conducted a user study with 12 interaction designers. As their tool, they used a custom-built virtual design environment with finger tracking and physics simulations for natural interactions with virtual devices and objects. Our study identified the designers' experience of space in relation to their own bodies and playful design explorations as key opportunities. Key challenges were the complexities of building a usable yet versatile VR-based "World Editor".
Research Questions / Practical Problems
Question signals indexed for this paper.
- 80%
Tesseract: Querying Spatial Design Recordings by Manipulating Worlds in Miniature
CHI '23· Mixed Reality Workspaces +1
- 67%
HOICraft: In-Situ VLM-based Authoring Tool for Part-Level Hand-Object Interaction Design in VR
CHI '26· Full-Body Interaction & Embodied Input +2
- 67%
Point & Grasp: Flexible Selection of Out-of-Reach Objects Through Probabilistic Cue Integration
CHI '26· Full-Body Interaction & Embodied Input +2
- 67%
GestureCanvas: A Programming by Demonstration System for Prototyping Compound Freehand Interaction in VR
UIST '23· Hand Gesture Recognition +2
- 67%
TwinSpin: A Virtual Ball in a VR Controller Enabling In-Hand 3DoF Rotation
UIST '25· Shape-Changing Interfaces & Soft Robotic Materials +2
- 60%
Evaluating the Combination of Visual Communication Cues for HMD-based Mixed Reality Remote Collaboration
CHI '19· Mixed Reality Workspaces
- 60%
Relative Design Acquisition: A Computational Approach for Creating Visual Interfaces to Steer User Choices
CHI '23· Computational Methods in HCI
- 60%
GraV: Grasp Volume Data for the Design of One-Handed XR Interfaces
DIS '24· Full-Body Interaction & Embodied Input +1
- 60%
DeepSI: Interactive Deep Learning for Semantic Interaction
IUI '21· Computational Methods in HCI
Based on Jaccard similarity of research subtopics & professions (≥60%)