Searching Through Complex Worlds: Visual Search and Spatial Regularity Memory in Mixed Reality

Paper Title

Searching Through Complex Worlds: Visual Search and Spatial Regularity Memory in Mixed Reality

Publication Info

• Topic area: Visual search and memory in mixed reality (MR) environments.
• Keywords: Visual search, spatial regularity, mixed reality, dual-task performance, physical environment complexity, virtual element depth, contextual cueing, workload, implicit memory, human-computer interaction.

Background and Problem

• Problem / challenge: Existing studies on visual search often focus on isolated factors (e.g., physical complexity or virtual layout) in 2D environments, neglecting the interplay of physical and virtual elements in MR. Additionally, the effects of dual-task demands and spatial regularities on search performance in MR remain underexplored.
• Significance: Understanding visual search and memory in MR is crucial for designing effective MR applications, as users often face complex environments and multitasking demands.
• Motivation and related work: Prior research has shown that physical and virtual elements influence visual search, but most studies were conducted on 2D displays or VR/AR, lacking depth-related insights. Studies on spatial regularities have also yielded inconsistent findings, particularly under dual-task conditions. This paper aims to address these gaps by examining how physical complexity, virtual depth, and dual-task demands interact in MR.

Solution

• Proposed approach: A user study using a custom MR application to systematically investigate the effects of physical environment complexity, virtual element depth, and dual-task presence on visual search performance and spatial regularity memory.
• Novelty:
  1. Disentangling the effects of physical environment complexity, virtual element depth, and dual-task presence on visual search in MR.
  2. Investigating the role of spatial regularities in MR, distinguishing between implicit and explicit memory.
  3. Highlighting the divergence between objective performance and subjective workload in MR tasks.
  4. Providing design recommendations for MR applications based on empirical findings.
• Procedure and key techniques:
  • Conducted a 2×2×2 within-subject experiment with 24 participants, manipulating physical environment complexity (simple vs. complex), virtual element depth (same vs. different), and task type (single vs. dual task).
  • Measured visual search performance (reaction times, accuracy), spatial regularity memory (contextual cueing and recognition tasks), and perceived workload (NASA-TLX questionnaire).
  • Analyzed data using Linear Mixed-Effects Models (LMMs), Generalized Linear Mixed-Effects Models (GLMMs), and Cumulative Link Mixed Models (CLMMs).

Results

• Concrete findings:
  • Complex environments significantly increased reaction times (β = 0.305, p < 9.65E-29) and reduced accuracy (OR = 0.716, p < 3.73E-04).
  • Virtual elements at different depths slowed reaction times (β = 0.091, p < 8.6E-4) but did not significantly affect accuracy.
  • Repeated spatial configurations reduced reaction times across all conditions, indicating implicit spatial regularity memory.
  • Dual-task presence did not significantly affect reaction times overall but increased perceived workload across all NASA-TLX subscales.
• Advantage over baselines:
  • Depth cues mitigated the negative impact of complex environments on reaction times (β = -0.110, p < 4.54E-3).
  • Repeated spatial configurations improved search efficiency even under dual-task and complex conditions.
• Experiments / evaluation:
  • Participants completed 125 trials of a contextual cueing task and 4 trials of a recognition task under 8 experimental conditions.
  • Reaction times, accuracy, and perceived workload were analyzed to evaluate the effects of the independent variables.
• Limitations and future work:
  • Controlled physical environments may not fully capture real-world MR variability.
  • Factors like virtual element color and size were not explored.
  • Sample size determination could benefit from a priori power analysis.
  • Future work should examine dynamic environments and additional factors influencing visual search and memory.

Summary

This study investigated how physical environment complexity, virtual element depth, and dual-task presence influence visual search performance and spatial regularity memory in MR. Results showed that complex environments and different-depth virtual elements impaired search efficiency, while repeated spatial configurations facilitated implicit memory-based improvements. Dual-task conditions increased perceived workload but had limited impact on objective performance. The findings highlight the importance of balancing user experience and performance in MR design, emphasizing the need for consistent spatial layouts and careful consideration of environmental and task-related factors. These insights provide actionable guidelines for optimizing MR applications in diverse contexts.