AReading with Smartphones: Understanding the Trade-offs between Enhanced Legibility and Display Switching Costs in Hybrid AR Interfaces

Research Background and Problem Statement

• What issues or challenges did the authors identify?

  • In optical see-through head-mounted displays (OST HMDs), text readability is limited by factors such as display resolution, transparency, high background light interference, and a restricted field of view. These issues significantly impact the user reading experience.
  • Using a smartphone as an auxiliary display could potentially compensate for the shortcomings of HMDs (e.g., improving display density and mitigating background interference), but it introduces additional costs related to visual attention switching. The authors aim to explore the specific impact of these costs on augmented reality (AR) reading tasks and the extent to which they disrupt the user experience.

• Why is this issue important?

  • Reading is a core task in many applications within augmented reality environments, such as museum guides, education, and virtual offices. Improving text readability in AR not only enhances user experience but also expands the potential application areas of the technology.

• Research Motivation and Related Work

  • Literature indicates that text reading efficiency in pure OST HMD environments is significantly lower than on traditional display devices. Some studies have proposed methods to optimize text design for OST HMDs, but few have focused on improving readability by integrating physical auxiliary devices such as smartphones.
  • By reviewing related research, the authors found that such integration introduces "display switching costs," such as visual fatigue, prolonged task completion time, and increased workload. However, the trade-off between these costs and the potential readability benefits remains underexplored.

Proposed Solution

• What methods or solutions did the authors propose?

  • A hybrid user interface (Hybrid User Interface) was proposed, combining OST HMDs with smartphones. The smartphone is used to display text, while the HMD displays virtual images.
  • The study evaluates the impact of this hybrid interface on AR reading tasks at different content distances through an experiment (N=24) to identify the balance between display switching costs and readability benefits.

• What is innovative about this solution?

  • This is the first systematic quantification and comparison of the performance, workload, fatigue levels, and visual behavior of a hybrid interface versus a single HMD interface in AR reading tasks.
  • The study provides design recommendations for optimizing hybrid interfaces, such as dynamically adjusting content distance and using smartphones as input devices.

• What are the implementation steps and key technologies used?

  • Experimental Design: The HMD provides virtual images, while the smartphone displays related text. Differences between the two interfaces are compared at various content distances (0.45m, 1m, 2m, 5m).
  • Key Technologies: The experiment utilized Microsoft HoloLens 2 and Samsung Galaxy S22 Ultra, with Unity and OptiTrack technology to monitor users' head and smartphone positions.
  • Data Collection: Metrics such as task completion time, error rate, workload (NASA TLX), visual and arm fatigue, as well as gaze and head movement behavior, were measured.

Research Findings

• What specific findings were achieved?

  • Task Performance: The hybrid interface reduced reading error rates across all distances, though it did not significantly improve task completion time.
  • Workload and Fatigue: The hybrid interface significantly alleviated visual fatigue, improved user ratings of text readability, and enhanced task focus. However, as content distance increased, arm fatigue became significantly more pronounced.
  • Visual Behavior: At greater distances (2m), the hybrid interface reduced the frequency of gaze switching between text and images. The increased display switching costs prompted participants to rely more on memory.

• What advantages does it have compared to existing solutions?

  • Compared to using only an HMD, the hybrid interface significantly improved text readability and task accuracy.
  • It mitigated common issues of visual fatigue in pure HMD environments, making it particularly suitable for text-intensive tasks.

• What were the experimental or evaluation results?

  • The results indicate that content distance has a significant moderating effect on interface effectiveness. At closer distances (0.45m), the hybrid interface demonstrated the most pronounced advantages, while at farther distances (5m), the costs of attention switching reduced its supportive role.

• Limitations and Future Directions

  • Limitations:
    • Differences in content layout under different interface conditions may have affected the standardization of the study.
    • The use of different text and background colors may have introduced subtle interference in the results.
    • Long-term behavioral adaptation and cumulative user fatigue effects were not observed.
  • Future Work:
    • Explore the applicability of the hybrid interface in other task scenarios (e.g., education, industrial applications).
    • Improve experimental design, such as using a unified color scheme or adding intermediate conditions.
    • Investigate the effects and behavioral changes of long-term hybrid interface use.

Through an in-depth analysis, the authors provided practical recommendations to further optimize hybrid user interfaces, offering clear directions for the broader implementation of AR reading applications.