MIRIA: A Mixed Reality Toolkit for the In-Situ Visualization and Analysis of Spatio-Temporal Interaction Data

Document Title

MIRIA: A Mixed Reality Toolkit for the In-Situ Visualization and Analysis of Spatio-Temporal Interaction Data

Document Information

• Domain: Mixed Reality and Visualization Analysis of Spatio-Temporal User Interaction Data
• Keywords: Interaction Analysis, Immersive Analytics, In-Situ Analysis, In-Situ Visualization, Augmented Reality, Human-Computer Interaction, Visualization

Research Background and Issues

• Identified Problems or Challenges:

  • Existing user interaction analysis tools are predominantly 2D, single-user, and confined to desktop environments, making it difficult to leverage the rich context of complex environments directly.
  • Multimodal interactions (e.g., user movement, mid-air gestures, touch, gaze) lack adequate tool support for analysis.
  • Analysis tools often fail to directly reflect the impact of spatial and environmental factors on user behavior.

• Significance:

  • Analyzing user interactions in mixed reality and multi-display environments is crucial for understanding user behavior in novel computing systems.
  • In-situ visualization integrates real-world spatial environments, aiding in the analysis of spatial interaction patterns and their cultural or physical influences.

• Research Motivation and Related Work:

  • Inspired by immersive analytics and multi-display environment research, the authors aim to design a mixed reality tool that supports in-situ, multi-user collaborative analysis.
  • They summarize common tasks, data types, and visualization methods of existing analysis systems, highlighting their limitations.

Solution

• Proposed Method or Solution:

  • Designed and implemented the MIRIA toolkit, which embeds AR visualizations into the physical space where data was recorded, enabling in-situ data exploration and analysis.
  • Provides various visualization formats (e.g., 3D motion trajectories, location heatmaps, scatter plots) and media integration (e.g., video and screenshots).

• Innovative Features:

  • Combines AR environments with user interaction data, supporting multi-user collaboration and enhanced spatial context.
  • Enables dynamic location display, flexible data filtering, and annotation functions, allowing researchers to analyze spatial interaction data more comprehensively and efficiently.

• Implementation Steps and Key Technologies:

  1. Data Import: User interaction logs are imported via CSV format, with metadata defined using XML files.
  2. Data Preprocessing: High-frequency tracking data is downsampled to improve computational efficiency.
  3. Visualization Generation: 3D and 2D visualizations are implemented using the Unity 3D engine.
  4. User Interaction: Multi-user exploration of in-situ data is supported, with various data filtering and annotation functionalities.

Research Outcomes

• Specific Results:

  • Proposed a technical concept for analyzing mixed reality and augmented reality spatial interaction data.
  • Developed a functional prototype system to support in-situ data visualization and collaborative analysis.
  • Demonstrated MIRIA's application across multiple real-world research datasets, including 3D spatial data analysis and multi-user interactive game behavior analysis.

• Advantages:

  • Compared to traditional tools, MIRIA better supports spatial context analysis, particularly for studying the impact of environmental conditions on behavior.
  • Supports multi-user collaboration, with flexible deployment of analysis tools that do not rely on additional tracking devices.

• Experiments or Evaluation Results:

  • Expert feedback indicates that MIRIA provides an innovative and effective analysis experience, including enhanced user engagement and improved reproduction of data context.
  • In-situ visualization excels in presenting event timelines and behavioral trajectories.

• Limitations and Future Directions:

  • Current hardware (e.g., HoloLens) limits large-scale data visualization.
  • Manually writing configuration files is cumbersome; future work may focus on automating data configuration tools.
  • Plans to enhance annotation functionality, support more complex filtering operations, and improve field-of-view and rendering quality.
  • Exploring tools to support more complex user analysis goals (e.g., displaying user behavior regions, gaze, and occlusion).