Uncertain Pointer: Situated Feedforward Visualizations for Ambiguity-Aware AR Target Selection

Paper Title

Uncertain Pointer: Situated Feedforward Visualizations for Ambiguity-Aware AR Target Selection

Publication Info

• Topic area: Augmented reality (AR) target selection and disambiguation through visual feedforward techniques.
• Keywords: Augmented reality, target disambiguation, feedforward visualization, uncertainty visualization, pointer design, user studies, systematic review, object selection, multimodal interaction.

Background and Problem

• Problem / challenge: AR systems often struggle with input ambiguity during target selection, especially in dynamic, cluttered, or distant scenarios. Existing visualizations for disambiguation are limited in scope and lack systematic evaluation across real-world complexities.
• Significance: Addressing input ambiguity in AR systems is critical for enhancing usability, reducing errors, and improving user confidence in multimodal interactions.
• Motivation and related work: Previous research has explored implicit and explicit disambiguation techniques, including visualizations like lassos and bounding boxes. However, these methods are rarely tested in varied AR contexts, leaving gaps in understanding their effectiveness across different target layouts and complexities.

Solution

• Proposed approach: Uncertain Pointer—a systematic exploration of feedforward visualizations that annotate multiple candidate targets in AR to convey system uncertainty and support disambiguation.
• Novelty:
  1. Development of a pointer space comprising 25 visualization designs based on archetypes, signifiers, and uncertainty complexity.
  2. Systematic evaluation of pointer designs through two online studies (n = 60 and 40) across varied target distances and sparsities.
  3. Design recommendations for selecting pointer types and signifiers based on AR context and task requirements.
• Procedure and key techniques:
  • Conducted a systematic literature review of visualization techniques from 30 years of research.
  • Generated a pointer space combining four archetypes (external, internal, boundary, fill) and four signifiers (color, size, opacity, text) with three uncertainty complexities (certain, identity, level).
  • Evaluated pointer designs through user studies measuring metrics like preference, confidence, mental ease, visibility, and error rates.

Results

• Concrete findings:
  • Boundary pointers consistently performed best for single-target annotation (certain pointers) in terms of preference, mental ease, and visibility.
  • External pointers were preferred for multi-target scenarios (identity and level pointers) due to their clarity and non-occlusion.
  • Text and size signifiers were most effective for conveying graded uncertainty in level pointers but introduced occlusion.
  • Uniform visualizations (none signifier) yielded higher accuracy in coarse selection tasks.
• Advantage over baselines:
  • Systematic comparison revealed trade-offs between visibility, clarity, and disambiguation efficiency across pointer types and signifiers.
  • Boundary pointers were superior for single-target tasks, while external pointers excelled in multi-target scenarios.
• Experiments / evaluation:
  • Study 1 (n = 60): Focused on certain and identity pointers across 168 video trials with varying archetypes, signifiers, and scenes.
  • Study 2 (n = 40): Evaluated level pointers across 108 video trials, introducing additional metrics like intuitiveness and error in identifying most/least certain targets.
• Limitations and future work:
  • Video-based studies may not fully replicate wearable AR experiences due to differences in motion, field of view, lighting, and depth cues.
  • The pointer space is not exhaustive; future work could explore additional signifiers and adaptive designs for larger candidate sets.
  • Integration with vision-language models (VLMs) and exploration of composable pointer designs are suggested.

Summary

Uncertain Pointer systematically investigates feedforward visualizations for AR target selection, addressing input ambiguity through novel pointer designs. Two user studies revealed key trade-offs in archetypes and signifiers, providing actionable recommendations for designing ambiguity-aware AR systems. The findings highlight the importance of balancing visibility, clarity, and disambiguation efficiency, with applications extending to human-robot interaction and multimodal AR interfaces. Future work could refine pointer designs for wearable AR and integrate them with advanced AI systems.