Every Move You Make: Visualizing Near-Future Motion Under Delay for Telerobotics

Paper Title

Every Move You Make: Visualizing Near-Future Motion Under Delay for Telerobotics

Publication Info

• Topic area: Predictive visualizations for mitigating operator uncertainty in delayed teleoperation.
• Keywords: Telerobotics, communication delay, predictive displays, trajectory visualization, feedforward interfaces, operator uncertainty, cognitive load, human-robot interaction, teleoperation interfaces, delayed feedback.

Background and Problem

• Problem / challenge: Communication delays in telerobotics decouple operator input from robot feedback, creating three distinct uncertainties: communication (timing of command execution), trajectory (mapping of inputs to motion), and environmental (external factors altering outcomes). Existing predictive displays often treat delay as a single issue and fail to address these uncertainties comprehensively.
• Significance: Effective teleoperation under delay is critical for high-stakes applications like planetary exploration and disaster response, where delays disrupt control and increase cognitive load, forcing operators into reactive behaviors that degrade performance.
• Motivation and related work: Prior work has shown that predictive displays can aid in anticipating delayed responses, but they rarely differentiate between uncertainty sources or incorporate probabilistic factors. This paper aims to address this gap by decomposing delay into its constituent uncertainties and evaluating visualization strategies tailored to each.

Solution

• Proposed approach: Development and evaluation of three predictive visualizations—Network, Path, and Envelope—that externalize communication, trajectory, and environmental uncertainties, respectively.
• Novelty:
  1. Decomposition of delay into three operator uncertainty facets: communication, trajectory, and environmental.
  2. Introduction of three visualization techniques targeting these facets: Network (command timing), Path (predicted motion), and Envelope (possible deviations).
  3. Empirical evaluation of these visualizations under a fixed 2.56 s round-trip delay in a controlled study.
  4. Evidence that trajectory-based visualization (Path) significantly improves performance and reduces cognitive load, unlike the other two approaches.
• Procedure and key techniques:
  1. Design of UNITE, a Unity-based simulation environment with configurable delay, noise, and terrain.
  2. Implementation of three visualizations:
     • Network: Displays timelines of command execution to externalize communication delay.
     • Path: Projects predicted motion trajectories to address trajectory uncertainty.
     • Envelope: Visualizes worst-case deviation bounds to communicate environmental uncertainty.
  3. Controlled study with 24 participants navigating a simulated robot under delay, comparing the visualizations against a delayed-video baseline.

Results

• Concrete findings:
  • Path visualization reduced task completion time (median 135.2 s vs. 209.9 s for baseline) and cognitive load (NASA-TLX mental demand score: 7.29 vs. 13.21 for baseline).
  • Envelope visualization reduced cognitive load (9.25) but did not improve task performance.
  • Network visualization showed no measurable improvement over the baseline.
• Advantage over baselines:
  • Path visualization enabled proactive control, reducing reliance on reactive “move-and-wait” behavior (median pauses: 1.0 vs. 5.0 for baseline).
  • Path was rated highest in perceived control, ease of interpretation, and reduced frustration.
• Experiments / evaluation:
  • Within-subjects design with 24 participants navigating a lunar-like terrain under a fixed 2.56 s delay.
  • Metrics: task completion time, cognitive load (NASA-TLX), reactive behavior, and subjective ratings.
  • Statistical analysis: Friedman tests with post-hoc Wilcoxon comparisons, mixed-effects modeling, and thematic qualitative analysis.
• Limitations and future work:
  • Fixed delay excludes variability seen in real-world networks.
  • Static terrain limits generalizability to dynamic environments.
  • Novice participants may not reflect expert operator behavior.
  • Future work should explore variable delays, dynamic environments, and long-term constructs like trust and situational awareness.

Summary

This study addresses the challenge of teleoperation under delay by decomposing operator uncertainty into communication, trajectory, and environmental facets and evaluating visualizations tailored to each. The Path visualization, which projects near-future motion, significantly improved task performance and reduced cognitive load, enabling proactive control. In contrast, the Network and Envelope visualizations provided useful information but failed to translate it into actionable control improvements. These findings highlight the importance of aligning predictive cues with the spatial decisions operators must make, offering a clear path forward for designing effective teleoperation interfaces under delay.