AvatAR: An Immersive Analysis Environment for Human Motion Data Combining Interactive 3D Avatars and Trajectories

Document Title

AvatAR: An Immersive Analysis Environment for Human Motion Data Combining Interactive 3D Avatars and Trajectories

Document Information

• Subject Area: Analysis and visualization of human motion data, studying human movement and interaction behaviors in augmented reality environments.
• Keywords: Human motion data analysis, augmented reality, mixed reality, interactive 3D avatars, planar and trajectory visualization, immersive analysis

Research Background and Problem

• Identified Issues and Challenges:

  • Current tools for human motion data analysis primarily rely on 2D or 3D charts (e.g., heatmaps, line graphs, trajectory maps) but lack visualization methods directly linked to real-world environments.
  • These traditional methods fail to provide detailed representations of human posture or interactions with the environment, making them suitable only for simpler spatial behavior or motion data analyses.
  • There is an urgent need for new technologies to bridge the gap between analysis tools and user data, enabling better understanding of human behavior.

• Importance of the Problem:

  • Human motion data reveals patterns of spatial usage, behavioral regularities, and the execution of complex workflows.
  • It is widely applied in areas such as office space optimization, dance learning, building energy efficiency management, and customer behavior analysis.
  • Conducting analysis in the actual environment where data is captured can provide a more direct, spatially aware research approach, enhancing usability.

• Research Motivation and Related Work:

  • The authors aim to improve the evaluation of human motion data by embedding virtual human representations and motion trajectories directly into real-world environments through an immersive augmented reality head-mounted display (AR HMD).
  • They propose a novel tool that combines traditional trajectory analysis with interactive virtual avatars to provide detailed information and enhance the analysis experience.

Solution

• Proposed Solution:

  • AvatAR System: An immersive analysis environment that leverages AR technology to embed human motion data into its corresponding real-world environment for on-site analysis.
  • Visualization Methods:
    • Interactive 3D avatars: Displaying individual postures and motion details.
    • Motion trajectories: 3D paths providing an overall spatial overview of movement.
    • Environment-embedded visualization: Including gaze visualization, touch state, and footprint visualization.

• Innovative Features:

  • Integrating motion data directly with real-world environments, presenting it not only as charts but also as interactive avatars and environmental interactions.
  • Enabling interaction through the avatar's body, such as accessing timelines, controlling visualization types, and manipulating playback.
  • Supporting diverse use cases, such as office space analysis, dance motion learning, and retail store layout optimization.

• Implementation Steps and Core Technologies:

  • Initialization and Data Integration: Parsing skeletal data from human motion datasets (e.g., OpenPose or CMU Panoptic Studio).
  • Generating 3D Avatars: Reconstructing postures based on data and matching the dimensions of the real-world environment.
  • AR HMD and Tablet Interaction: HMD for immersive experiences, with tablets displaying floor plans to assist navigation and visualization selection.
  • Environment Embedding: Generating additional environmental data points (e.g., touch, gaze) by calculating interactions with real objects.

Research Outcomes

• Specific Outcomes:

  • Developed the AvatAR prototype, supporting real-time generation of 3D avatars and related trajectories, integrated into AR for advanced human behavior analysis.
  • Demonstrated the system's application potential in various scenarios, such as office analysis, dance learning, and customer behavior research.

• Comparison with Existing Solutions:

  • Compared to traditional 2D or 3D analysis tools, AvatAR adds immersive environment support and detailed dynamic posture reconstruction.
  • Surpasses similar tools (e.g., MIRIA) in capabilities such as embedding direct environmental interactions and detailed reconstruction.

• Experimental or Evaluation Results:

  • Prototype testing indicates that AvatAR helps users better understand human motion data, particularly in tasks involving environmental context.
  • Simulated scenarios showcased its potential applications in optimizing spatial layouts (offices or stores), guiding dance movements, and recording customer behavior.

• Limitations and Future Directions:

  • Currently lacks detailed testing for longer time periods or larger-scale datasets.
  • Some technologies are not yet optimized for higher data precision (e.g., eye-tracking information).
  • Large-scale user evaluations have not been conducted; future research should focus on field studies to validate the system's effectiveness in supporting analysis tasks.

Conclusion

The AvatAR system demonstrates new potential for immersive analysis of human motion data. By combining 3D avatars, trajectory management, and environment-embedded visualization with real-time AR technology, it enables not only exploration but also experiential interaction with data. This enhanced immersion and perspective transformation are expected to play a significant role in analysis tasks across multiple domains.