Augmented Journeys: Interactive Points of Interest for In-Car Augmented Reality

Research Background and Issues

• What problems or challenges did the authors identify?
  This study focuses on how passengers in vehicles can use augmented reality (AR) technology to interact with and explore fixed points of interest (POIs) in the surrounding environment. The effectiveness of existing interaction techniques and visualization methods in dynamic vehicle environments remains unclear. Additionally, passengers face difficulties in recalling the names of missed locations and exploring the surrounding environment from within the vehicle.

• Why is this issue important?
  With the increasing prevalence of autonomous vehicles, the amount of time people spend as passengers in vehicles is steadily growing. This time is often perceived as unproductive or wasted. Transforming travel time into meaningful interaction or entertainment experiences could significantly enhance user satisfaction.

• Research Motivation and Related Work
  The motivation stems from passengers' need to explore surrounding POIs, such as restaurants, parks, and museums, while increasing awareness of and access to information about these locations. Related work involves the application of AR technology in vehicle environments, but existing research primarily focuses on interactions with in-car elements or interactions in static environments.

Solution

• What methods or solutions did the authors propose?
  The authors designed and evaluated an AR system based on a video see-through head-mounted display (VST HMD) that allows passengers to interact with POIs using eye tracking and pinch gestures. They also designed three visualization methods (list, timeline, and minimap) to help users browse upcoming or missed POIs.

• What are the innovative aspects of this solution?
  Innovations include:

  1. Utilizing AR technology to enable interaction between world-fixed POIs and in-car UI in a dynamic environment.
  2. Designing three unique visualization methods to support POI exploration.
  3. Experimentally validating the feasibility of eye tracking and pinch gestures for interaction in dynamic vehicle environments.
  4. Providing detailed POI information, including names, categories, and descriptions, to help users quickly identify locations.

• What are the implementation steps and key technologies used?

  1. System Design: Developed an AR display of POIs using Unity and the Varjo XR-3 video see-through head-mounted display.
  2. Interaction Technology: Used eye tracking for pre-selection and pinch gestures for confirmation and scrolling.
  3. Visualization Design: Designed list, timeline, and minimap as in-car UIs to support exploration of past and future POIs.
  4. User Study: Conducted a three-phase experiment, including surveys, pilot studies, and main experiments, to investigate passenger habits, interaction behaviors, and system acceptance.

Research Outcomes

• What specific outcomes were achieved?

  1. Reported passenger and driver needs and basic habits in saving and locating POIs, including a preference for manually saving points of interest.
  2. Found that eye tracking combined with hardware buttons is suitable for AR interaction in dynamic vehicles, while pinch gesture interaction showed limited effectiveness.
  3. Identified the list visualization as the most preferred method due to its familiarity, ease of search, and minimal impact on external visibility.
  4. Demonstrated the system's effectiveness in addressing the challenge of identifying missed locations and highlighted the need for lightweight and comfortable hardware.

• What advantages does it have compared to existing solutions?

  1. Provides a unique solution for interacting with the external environment in dynamic traffic scenarios.
  2. Optimizes POI information display and interaction methods based on user needs.
  3. System design adheres to principles of enhancing user experience, such as reducing cognitive load through information recognition.

• What were the experimental or evaluation results?

  1. Workload: The perceived workload of using the minimap was significantly higher than that of the list and timeline methods.
  2. User Experience: The list method received the highest ratings, excelling in attractiveness, novelty, and ease of use among the three visualization methods.
  3. Task Completion Rate: The task completion rates for the list and timeline methods were close to 90%, significantly outperforming the minimap's approximately 60% completion rate.
  4. Mobile Device Limitations: Current hardware is heavy and generates heat, reducing user experience. Additionally, vehicle vibrations affected interaction accuracy.

• Limitations and Future Directions

  • Limitations:

    1. The hardware used in this study (Varjo XR-3) is heavy and unsuitable for prolonged wear.
    2. Pinch gesture interaction performed poorly in unstable vehicle environments.
    3. The experimental environment was limited to urban industrial areas, which may not generalize to other scenarios.
    4. The sample group was biased toward employees of automotive software companies, potentially affecting the results.

  • Future Directions:

    1. Explore alternative interaction methods, such as integrating voice input or using handheld devices.
    2. Optimize POI localization techniques in dynamic environments, for example, by anchoring POIs to buildings using computer vision.
    3. Develop lighter AR glasses or integrate the system into in-car HUDs.
    4. Expand research scenarios to various driving contexts, such as daily commutes and long-distance travel.