DistanciAR: Authoring Site-Specific Augmented Reality Experiences for Remote Environments

Document Title

DistanciAR: Authoring Site-Specific Augmented Reality Experiences for Remote Environments

Document Information

• Subject Area: Augmented Reality (AR), Remote Virtual Environment Design
• Keywords: Augmented Reality, Remote Authoring, Spatial Design, 3D Scanning, User Interface, Remote Environment, AR Tools, Mixed Reality

Research Background and Issues

• Problems and Challenges:

  • Traditional augmented reality (AR) authoring tools are typically limited to the designer's current physical environment. In the context of modern globalization and distributed work, designers need to create AR experiences tailored to remote environments.
  • The primary challenge of remote AR design lies in the designer's inability to directly access remote scenes, making it difficult to understand the spatial layout, dimensions, and content of the environment.
  • Existing tools, while offering desktop previews and generalized scene support, fail to meet the site-specific contextual needs of remote environments.

• Significance and Research Motivation:

  • Remote AR authoring tools can overcome geographical limitations, providing greater flexibility and convenience in applications such as interior design, remote education, and navigation.
  • Particularly during global pandemics, AR tools supporting remote authoring can meet the demands of remote collaboration and asynchronous design.

Solution

• Method or Approach:

  • A tablet-based remote AR authoring workflow called DistanciAR is proposed, consisting of three steps:
    1. Scan: Use LiDAR-equipped devices (e.g., iPad) to scan remote environments.
    2. Author: Simulate the designer's AR authoring experience for remote scenes from different locations.
    3. View: Visualize the final design in the original scene.

• Innovative Aspects of the Solution:

  • Two new modes are introduced:
    1. Dollhouse Mode: Provides a "god's eye view" of the remote scene, allowing designers to quickly understand the overall layout and navigate the remote scene efficiently.
    2. Peek Mode: Generates realistic composite images using captured video frames, offering more authentic visual feedback to enhance the authoring experience when 3D scan quality is suboptimal.
  • The existing Portal mode is improved to provide a more seamless and localized AR tool interaction experience.

• Key Implementation Steps and Techniques:

  • Scan: Use the iPad's LiDAR to generate 3D meshes of remote environments, combined with video recording and ARKit to capture precise positional information.
  • Dollhouse Mode Design: Implement single-sided material rendering, allowing users to quickly switch positions and perspectives through orbital camera interactions.
  • Peek Mode Design: Dynamically capture the video frame closest to the current perspective by comparing the UV coverage of the AR camera and the scanning recording camera.
  • Enhanced Interaction Methods: Add advanced navigation and object manipulation features, including visual feedback (highlighting, localized point cloud visualization) and group object operations.

Research Results

• Experiments and Evaluations:

  • First Round of Exploratory Studies:
    • User feedback indicated that while the foundational system of DistanciAR effectively addressed some remote AR authoring needs, improvements were needed in navigation and understanding remote environments.
  • Improved User Studies:
    • Compared to the version with only Portal mode, the complete system with Dollhouse and Peek modes received better user evaluations, showing stronger performance in navigation (Q3, p=0.0004, t=-5.438) and understanding remote environments (Q2, p=0.003, t=-4.019).
    • Timing and task analysis revealed that the complete system enabled more efficient task execution and more accurate operations. For example, Dollhouse mode was used for global navigation, while Peek mode facilitated fine-tuning and design verification.

• Comparison with Existing Solutions:

  • DistanciAR combines the immersive attributes of mobile AR tools with the global environmental information of desktop tools, bridging the functional gaps between the two.
  • The rarely used non-immersive mode (Dollhouse) in AR authoring demonstrated significant value.

• Limitations and Future Directions:

  • Scanning Quality Issues: The current quality of 3D scanning and texture mapping still impacts the placement of virtual objects. Future work could optimize scanning through enhanced machine learning algorithms.
  • User Experience Improvements: Peek mode lacks independent depth information support, resulting in incomplete occlusion effects. Additionally, multi-view switching may increase complexity.
  • Expansion and Scene Adaptation: Future research could explore multi-user collaboration modes, emphasize interactive behaviors in real-world environments, and support more diverse themes (e.g., dynamic triggers or animations) in AR functionality.

Conclusion

DistanciAR proposes an innovative remote AR authoring workflow, particularly suited for remote environments and multifunctional authoring needs. While its core innovations address many limitations of existing methods, there is still room for further optimization, especially in areas such as 3D scanning quality and composite depth mapping techniques. This research lays a solid foundation for future studies on cross-environment AR authoring and collaboration.