Data Cubes in Hand: A Design Space of Tangible Cubes for Visualizing 3D Spatio-Temporal Data in Mixed Reality

Document Title

"Data Cubes in Hand: A Design Space of Tangible Cubes for Visualizing 3D Spatio-Temporal Data in Mixed Reality"

Document Information

• Subject Area: Tangible User Interfaces (TUI) and data visualization design in Mixed Reality (MR) environments
• Keywords: Tangible cubes, mixed reality, spatio-temporal data, data visualization, user interaction, design space, prototype evaluation, immersive environments, interaction mapping, augmented reality

Research Background and Issues

• Problems and Challenges:

  1. Tangible interfaces in Mixed Reality (MR) environments offer intuitive interaction advantages, but the design potential of cube-shaped objects remains underexplored.
  2. Visualizing multidimensional data, especially spatio-temporal data, is complex and difficult to comprehend, with existing technologies and methods facing limitations.
  3. There is a lack of a unified framework to explore all possibilities of tangible cube design and interaction.

• Research Significance:

  • Tangible cubes not only provide a natural model for three-dimensional interaction but are also particularly suitable for representing complex spatio-temporal data due to their modular structure.
  • Spatio-temporal data has practical applications in various fields (e.g., historical evolution, motion trajectory prediction, and natural disaster analysis) and exhibits greater expressive potential in MR environments.

• Research Motivation and Related Work:

  • Based on a literature review, the authors found that while some studies combine tangible interfaces with data visualization, they do not systematically explore the dimensions of interaction space, visualization space, and other possibilities of cubes.
  • Existing methods primarily focus on single-use cases and have not formed a unified design space or theoretical framework.

Solution

• Proposed Method: Establish a design space for tangible cubes to explore their interaction and visualization potential in mixed reality.

  1. The design space includes four components: size, interaction space, visualization space, and diversity.
  2. Conduct workshops (N=24) to collect user suggestions on different interaction tasks and refine specific interaction-visualization mappings.
  3. Develop prototype examples to validate the practical applicability of the design space and conduct user testing.

• Innovative Aspects of the Solution:

  • Defined a user-driven mapping relationship integrating interaction behaviors and visualization commands.
  • Provided a systematic framework to support visualization design for multidimensional data exploration using cubes in MR environments.
  • Implemented two visualization styles—dynamic (updated with movement) and anchored (fixed reference)—to meet different observation needs.

• Implementation Steps and Techniques:

  1. Preliminary Research: Literature evaluation and analysis of existing tangible cube-based interaction designs.
  2. Design Space Division: Proposed an interaction-visualization mapping framework based on interaction possibilities (e.g., rotation, stacking), visualization presentation methods (e.g., overlay, hiding, reconfiguration), and diverse layouts.
  3. User Workshops: Collected and analyzed 295 interaction-visualization mappings, refining interaction action categories (e.g., single-point touch gestures, multi-point interactions, surface trajectories).
  4. Prototype Creation: Developed a spatio-temporal data visualization prototype system based on selected interaction groups, utilizing Unity and Hololens hardware to enhance user experience.
  5. User Experiment Validation: Conducted qualitative evaluations of usability with expert and intermediate user groups.

Research Outcomes

• Specific Outcomes:

  1. Constructed a user behavior-based design space for tangible cubes, encompassing a comprehensive mapping of interaction actions (over 12 types) and visualization commands (data transformation, visual transformation, process control).
  2. Developed a prototype system, including physical modular cubes and an MR interface, to demonstrate global health expenditure data.
  3. Defined two distinct visualization styles—anchored and dynamic—making interactions both engaging and efficient.

• Advantages Compared to Existing Solutions:

  • Offers a more intuitive and immersive data interaction method compared to traditional flat displays.
  • Emphasizes flexibility and user-centric data exploration, supporting multiple interaction points and combination schemes.
  • Closely integrates users' intuitive actions (e.g., flipping, stacking) with complex visual representations.

• Experiment or Evaluation Results:

  • Users found the cubes highly manipulable, with intuitive actions enabling tangible data exploration.
  • The dynamic and anchored visualization modes demonstrated complementary effects in exploration and understanding: dynamic modes enhanced exploratory engagement, while anchored modes provided a stable baseline for comprehension.
  • Combining multiple cubes significantly improved comparative analysis capabilities and was easy to learn and use.

• Limitations and Future Directions:

  • Limitations:
    1. The study primarily focused on small and medium-sized cubes, without fully exploring the design of large cubes.
    2. Multi-cube systems may impose cognitive load on users, such as confusion arising from increased data complexity, which needs further resolution.
  • Future Directions:
    1. Explore extended models incorporating more interaction dynamics and visual styles.
    2. Continuously optimize material choices, such as adding tactile feedback with flexible materials.
    3. Validate the framework's applicability in different scenarios (e.g., education, weather analysis).
    4. Address multi-cube tracking and data synchronization challenges using more efficient technologies (e.g., optical or inertial tracking).

Conclusion

Through design, prototyping, and user research, this study proposes a systematic design space for tangible cube-based data interaction in mixed reality environments. This framework serves as a foundation for future MR data visualization design, providing a new methodological approach for multidimensional data analysis while emphasizing user-driven design, flexibility, and immersion.