Comparison of Different Types of Augmented Reality Visualizations for Instructions

Document Title

Comparison of Different Types of Augmented Reality Visualizations for Instructions

Document Information

• Subject Area: Application of Augmented Reality (AR) technology in industrial scenarios, focusing on the impact of visualization methods on user performance
• Keywords: Augmented Reality, Visualization, User Study, Instructional Design, Task Completion Time, Error Rate, Cognitive Load

Research Background and Issues

• Issues and Challenges:

  • As the application of AR in industrial guidance and remote maintenance increases, the impact of different types of visualization content on user performance during complex task execution remains unclear.
  • Existing studies mostly compare different media (e.g., paper instructions vs. AR) or hardware (e.g., head-mounted displays vs. handheld devices), with limited research on comparing visualization formats on the same hardware.
  • The relationship between complex AR content design and user cognitive load has not been clearly defined.

• Significance:

  • Understanding how AR visualization content affects user performance is crucial for improving user experience and optimizing industrial applications, especially for complex tasks such as mechanical debugging.
  • During the COVID-19 pandemic, AR technology has proven effective in enhancing remote maintenance, ensuring production continuity under pandemic conditions.

• Research Motivation and Objectives:

  • Conduct empirical research to clarify the impact of different AR visualization formats on task completion time, error rate, and user cognitive load during complex mechanical debugging tasks.
  • Investigate the performance of specific (3D CAD models), abstract (simple arrows and frames), and hybrid formats combining video content.

Solution

• Proposed Method:

  • Conduct a user study using head-mounted displays (Microsoft HoloLens) to compare five visualization formats:
    1. Abstract AR (AAR): Using simple 3D arrows and frames.
    2. Concrete AR (CAR): Using detailed 3D models derived from CAD.
    3. Abstract AR + Video (AAR+V): Adding video guidance to abstract AR.
    4. Concrete AR + Video (CAR+V): Adding video guidance to concrete AR.
    5. Paper Instructions: Used as a baseline reference.

• Core Innovation:

  • Systematically compare the impact of different AR visualization formats on user performance, particularly the potential reduction in cognitive load and improvement in task completion efficiency when incorporating video.
  • Provide experimental scenarios based on real industrial environments, simulating the complexity of mechanical assembly tasks.

• Key Techniques and Steps:

  1. Data Recording and Collection:

     • Use Microsoft HoloLens to record task completion time, time spent on each step, and video viewing frequency.
     • Manually tally task error rates.
     • Administer standardized questionnaires on task load, cognitive load, and user experience (RSME, NASA-TLX, SUS, UEQ).

  2. User Sample:

     • Recruit 48 participants, including students and professionals from various industries, with no prior mechanical operation experience.

  3. Experimental Design:

     • Five experimental conditions (five instruction formats), with no crossover between groups to avoid learning effects.
     • Tasks involve installing complex mechanical tools with predefined step sequences, some steps being highly complex.

  4. Data Analysis Methods:

     • Use Analysis of Variance (ANOVA) to evaluate significant differences in task completion time and task load.
     • Use Kruskal-Wallis tests to analyze error rates, supplemented by paired post-hoc analyses.

Research Findings

• Overall Findings:

  • Task Completion Time: No significant differences were observed between visualization formats. While video did not significantly reduce completion time, it provided additional process safety.
  • Error Rate:
    • Concrete visualizations (CAR) and video-enhanced formats (CAR+V, AAR+V) significantly reduced task error rates.
    • Pure abstract visualization (AAR) resulted in the highest error rates.
  • Cognitive Load:
    • Compared to paper instructions, abstract visualizations increased user cognitive load.
    • Video significantly reduced cognitive and mental stress during tasks.

• User Experience Evaluation:

  • All forms of AR content demonstrated good usability (SUS scores > 68).
  • Users generally found AR more intuitive, with video-enhanced instructions boosting operational confidence.

• Limitations & Future Directions:

  • Sample Limitation: The structure of the 48-person sample may limit the generalizability of the results.
  • The current study did not systematically investigate the effectiveness of video as a standalone condition.
  • Future research could focus on:
    • Differences in AR design needs between domain experts and general users.
    • The potential impact of multimodal interactions (gestures, voice, etc.) on user experience.
    • Balancing content complexity with cost-effective creation.

Summary and Conclusion

• Concrete AR (CAR) is suitable for complex, high-precision tasks, while abstract AR (AAR) is better for scenarios requiring rapid AR content generation.
• Video content significantly complements AR visualizations, enhancing user experience and task quality regardless of whether the format is abstract or concrete.
• In industrial applications, AR design should be tailored to task complexity, with recommendations to combine concrete formats with multimedia content for complex tasks.