Missing haptics require a visual substitute
Aliases: visual substitution for missing haptics · telerobotics
What it is
When a teleoperation system lacks force-feedback hardware, contact information that would otherwise be carried by touch needs to be conveyed through visual substitution — a force-magnitude bar, color-coded contact state, or a rendered visualization of deformation.
Why it happens
Visual substitution is workable but imperfect because converting haptic information into a visual symbol loses the directness and high bandwidth of touch itself — haptic feedback is a continuous physical quantity sensed directly by the operator's hand muscles, fast and not competing for visual attention, whereas a visual substitute requires the operator to actively look away from the task view to read an extra chart or indicator, consuming already-scarce visual attention and adding a reading delay. This means visual substitution is always a second-best option, used only when hardware cost or system complexity rules out force feedback — it closes the gap between "no contact information at all" and "some contact information," not the gap between "visual substitution" and "real force feedback." Auditory substitution — encoding force magnitude as pitch or a cueing tone — is another non-haptic channel option; compared with visual substitution it does not compete for visual attention, but it carries a more limited information dimension, usually able to encode only a single scalar rather than direction and magnitude together the way vision can, so the two are complementary rather than mutually exclusive and can be combined as the task requires.
Studying it
A common design compares task performance across three conditions — no contact information, visual substitution, and real force feedback — to see how much of the gap to real force feedback visual substitution actually closes, and to compare which visual encoding (numeric readout, color gradient, deformation animation) operators can read fastest without disrupting attention to the primary task. This line of work also typically checks whether the displayed force value is reliable in the first place — in many systems the "force" shown is a model estimate rather than a direct measurement, and estimation error directly undermines how trustworthy the substitute display is.
Where it stops holding
How effective visual substitution is depends on how much visual attention the primary task already consumes — if the primary task itself demands high visual focus (fine navigation, for instance), adding another indicator that needs to be read can distract rather than help. This approach suits tasks with comparatively light visual load and spare attentional capacity better. Color-vision deficiency, display latency, and the operator's gaze leaving the screen all break visual substitution outright — these are its fundamental weaknesses relative to haptic feedback, which does not compete for the visual channel at all. Head-mounted displays and desktop monitors also differ here: a headset's own field of view is already limited, so overlaying an extra force indicator is more likely to occlude the task view, whereas a desktop monitor usually has more spare screen margin available to hold such an indicator.
Applying it
In systems where budget or hardware rules out force feedback, prioritize substitutes that do not occupy central visual attention — a color change at the edge of the frame, an indicator perceptible in peripheral vision — over a chart that requires dedicated reading, and place the contact-state indicator where the operator's gaze naturally passes. Label clearly on the interface whether a shown force value is measured or model-estimated, so operators do not mistake estimation error for real contact state. Validate by testing whether operator performance on contact-related tasks improves after adding the visual substitute, while using eye tracking or a dual-task paradigm to check whether the indicator is significantly pulling visual attention away from the primary task.