Traversing Dual Realities: Investigating Techniques for Transitioning 3D Objects between Desktop and Augmented Reality Environments

Research Background and Problem

• Identified Problems or Challenges:
  The authors observed that seamlessly transferring 3D objects between desktop environments and augmented reality (AR) environments poses a significant challenge. Although some related technologies and methods exist, their effectiveness and application in real-world scenarios have not been systematically evaluated.

• Importance of the Problem:
  In modern office settings, the desktop remains central to most knowledge work demands. However, AR offers unique advantages, such as handling complex 3D information and enhancing multidimensional data interaction experiences. Effectively integrating desktop and AR environments could open new possibilities for scientific visualization and knowledge work paradigms.

• Research Motivation and Related Work:
  Many existing studies have explored the integration of desktop and AR to support 2D and 3D tasks. However, mechanisms that allow users to freely transfer digital objects between these two environments still have room for improvement, particularly in reducing cognitive load and improving workflow efficiency. Other works have proposed design spaces for cross-device interaction or single-user cross-reality tasks, but most lack concrete evaluation.

Solution

• Proposed Solution:
  The authors developed a series of techniques for transferring digital objects between desktop and AR environments. Initially, they proposed three basic technique pairs based on standard interaction methods: mouse/keyboard, close-range grabbing gestures, and mid-range grabbing gestures. Additionally, through user studies and feedback, they proposed optimized versions, such as "throw" and "catch" gestures.

• Innovative Aspects of the Solution:
  Key innovations include:

  1. Proposing a systematic set of transfer technique pairs, covering both desktop-to-AR and AR-to-desktop bidirectional transfers.
  2. Leveraging the natural advantages of gesture interaction in AR environments and integrating various distance-range interaction techniques to enhance flexibility.
  3. Allowing users to customize animation parameters, such as position transformation duration, rotation, and scaling adjustments, providing high adaptability to real-world scenarios.

• Implementation Steps and Key Techniques:

  1. Developing a cross-reality prototype system consisting of a desktop client and an AR client, enabling metadata and digital object interaction via APIs.
  2. Designing three basic transfer technique pairs, including "button press," "close-range grab," and "long-range grab."
  3. Providing dynamic adjustment features, enabling users to customize animation duration and end states.
  4. Collecting behavioral and subjective preference feedback through user studies and proposing improved versions, such as "catch" and "throw," in subsequent iterations.

Research Outcomes

• Specific Achievements:

  1. Foundational user studies revealed that "close-range grab" was the most favored technique, while "long-range grab" and button-based techniques were practical but slightly cumbersome.
  2. The optimized techniques (e.g., "throw" and "catch") were highly appreciated by expert users, particularly in scenarios involving the analysis of complex molecular structures.
  3. Validation of the techniques in real workflows with expert chemists demonstrated that the transfer techniques significantly reduced cognitive load during the transfer process and improved workflow efficiency.

• Comparison with Existing Solutions:
  Compared to existing methods, this study not only provides clear transfer mechanisms but also effectively evaluates their applicability through experiments, especially in real-world tasks. Many existing works merely propose concepts without practical validation.

• Experimental or Evaluation Results:
  In the first user study, it was found that:

  • Gestures were considered the most intuitive and easy-to-learn interaction method.
  • Animations were deemed important only for longer distances, while rotation and scaling enhancements were relatively overlooked.
    In the second expert study, it was found that:
  • Experts preferred using multiple transfer techniques to accommodate different scenario requirements and highly appreciated the naturalness and efficiency of gestures.
  • Customizable animation settings further enhanced adaptability to user needs.

• Limitations and Future Directions:
  Limitations include:

  1. Gender imbalance among participants in the experiments may affect the generalizability of the results.
  2. The user studies were based on small sample sizes, requiring larger-scale experiments to support statistical inference.
  3. The integration of 2D and 3D interactions in complex interfaces has not been deeply explored, necessitating future research on cross-content transfers.

  Future directions:

  • Exploring collaborative transfer techniques in multi-user scenarios.
  • Optimizing device selection in cross-reality environments (e.g., improved AR display devices).
  • Investigating the adaptability of transfer techniques in other fields, such as architectural design and data visualization.

Through systematic research and experimental evaluation, this paper makes substantial contributions to the problem of transferring digital objects between desktop and AR environments, while also providing design references and research directions for future work in related fields.