"Through the Looking Glass, and What We Found There": A Comprehensive Study of User Experiences with Pass-Through Devices in Everyday Activities

Research Background and Issues

• Issues or Challenges: Although mixed reality (MR) devices, such as video pass-through devices, are increasingly being applied in fields like education and healthcare, there remains a research gap in their integration into daily activities. Specifically, little is known about how these devices impact user experiences in everyday tasks, such as spatial perception, cognitive load, and social interaction.
• Significance: Investigating the impact of these devices in everyday scenarios can advance the widespread adoption of MR systems in the future, improving user experience, technical safety, and social acceptance.
• Research Motivation and Related Work: While previous studies have explored the application of MR devices in gaming, fitness, and desktop applications, most have been limited to laboratory settings or lacked multidimensional systematic analysis. To address this gap, this study focuses on the potential challenges and opportunities of MR devices in real-world daily tasks.

Solution

• Research Methodology: The authors designed a comprehensive field study, setting up three dimensions: degree of movement (dynamic vs. static tasks), physical vs. virtual interaction elements, and indoor vs. outdoor environments. Using a 2×2×2 experimental design, they tested 8 daily activity scenarios with 16 participants.
• Innovations:
  1. Introducing and integrating multiple measurement methods, including questionnaires (e.g., NASA-TLX, ARI, CAQ), biometric data collection (e.g., heart rate and electrodermal activity), and topic analysis based on language models.
  2. Conducting a comprehensive analysis of user experiences across multidimensional tasks in relation to the technical limitations of MR devices.
• Implementation Steps:
  1. Task Design: Define 8 tasks, such as ordering at a café (static/physical/indoor), recycling sorting (static/physical/outdoor), and watching videos (static/virtual/outdoor).
  2. Data Collection: Gather quantitative data (questionnaires, heart rate, electrodermal activity) and qualitative feedback from user interviews.
  3. Analysis Dimensions: Compare user experience differences across task dimensions using paired tests or variance analysis.
  4. Thematic Analysis: Use large language models (GPT-4) to assist in coding and extracting themes from qualitative data, uncovering patterns in complex user feedback.

Research Findings

• Specific Findings:
  1. Dynamic vs. Static Tasks: Dynamic tasks increased collision anxiety and cognitive load but did not significantly reduce immersion.
  2. Physical vs. Virtual Interaction: Virtual tasks enhanced immersion and reduced cognitive load, while physical tasks resulted in higher motion sickness (e.g., visual distortion).
  3. Indoor vs. Outdoor Environments: Outdoor tasks induced greater social anxiety and physical burden, though immersion and spatial disorientation did not significantly increase.
  4. Task Preferences: Video watching and fitness-related tasks were most favored by participants, while text input tasks (e.g., sending messages) received poorer feedback.
• Experimental Results:
  • Participants reported significant subjective differences across tasks. For instance, dynamic tasks (e.g., outdoor navigation) led to higher heart rate and electrodermal activity, while static video watching provided a relaxing and enjoyable experience.
  • Thematic analysis assisted by large language models identified 10 major themes and 26 sub-themes, highlighting core issues such as visual experience, interaction challenges, immersion, and social acceptance.
• Advantages:
  • The study provided a comprehensive set of cross-dimensional use cases, thoroughly covering the potential strengths and weaknesses of the technology.
  • The integration of advanced biometric signals and natural language processing enhanced the depth and efficiency of the analysis.
• Limitations and Future Directions:
  1. The small sample size (16 participants), with a bias toward younger individuals and those interested in IT, limits generalizability.
  2. The short duration of tasks (approximately 8 minutes per task) may not fully reflect the long-term effects of device usage.
  3. Outdoor tasks were conducted under controlled conditions, without accounting for variables such as weather and time of day.
  4. The need for improvements in physical device design (e.g., weight, field of view) remains insufficiently addressed.

Future research could explore:

• Testing devices with a larger and more diverse participant pool.
• Validating more complex, longer, and natural task settings.
• Investigating the impact of dynamic environmental variables, such as weather and day-night cycles, on device usage.
• Combining hardware and software improvements to enhance social acceptance and expand application scenarios.

In summary, this study provides valuable insights into the potential and challenges of MR technology, represented by video pass-through devices, in daily life. It also paves the way for future technological optimization and broader societal adoption.