Vision and sound can fake a touch that is not there

What it is

A blade meets a shield: sparks and a metallic crack arrive together, and the sword hand met nothing. Pseudo-haptics is vision and audition writing a contact that did not occur. No force arrived at the skin; the frame is still parsed as a hit. What is being forged is the sensation of touch, not a substitute commit for a button.

Why it happens

In cue combination, vision captures “did contact happen.” If the object stops on the frame the hand arrives, the surface dents, particles scatter, and a short impact lines up in time, contact is attributed — even as proprioception says the arm is still travelling. Control-display ratio goes one layer down: a cursor that suddenly slows on a “surface” is read as stiffness while the handheld torque stays zero. That is the Lécuyer move: an anomaly in visual motion standing in for force.

The illusion has a budget. Once the hand is plainly through the table, or the head is still walking into the wall, the proprioceptive contradiction cannot be attributed and the fake touch collapses into clipping. Sound alone is weaker than picture, but it can tip an ambiguous visual graze into a hit; desynchronised sound does the reverse and makes contact feel faker than silence.

Studying it

The classic manipulation changes C/D ratio so a cursor slows on a virtual surface, then collects perceived stiffness. A second family is the collision event: impact sound on or off, object stops or keeps penetrating, “did you feel a hit?” Temporal alignment is a hidden factor — tens of milliseconds of audiovisual offset drop the illusion.

Independent variables: surface deformation, impact sound, audiovisual offset, C/D ratio, penetration depth. Dependent variables: reported contact, perceived stiffness, “it felt like a real hit,” depth at which clipping is noticed.

Keep “it felt stiff” apart from “I know I pressed.” The first is pseudo-haptics; the second is commit confirmation. Do not put them on one scale.

Where it stops holding

A grounded force device is already producing force; an exaggerated visual dent then conflicts rather than fills in. Reaches across the midline, or targets in the far periphery, weaken visual capture and leave sound holding a budget it does not have. Silent use, or deaf users, lose half the channel. First-time wearers credit sparks as contact; practised users watch their own wrists, and once clipping is stared at, the fake touch breaks first. Optical see-through shows the real empty hand in real empty space, so the window for a forged contact is narrower than in video see-through.

Applying it

• On the contact frame, fire an aligned short impact and a surface reaction (stop, dent, particles). Do not let objects pass through the hand in silence.
• Use C/D ratio to make a drag feel sticky or springy. Do not ask it to timestamp a precise commit.
• Keep sound and picture aligned at frame scale. A late echo turns a collision into a replay.
• How to check: leave rumble off (it is often off already). Toggle only impact sound and deformation, and ask “did this one make contact?” Contact with the cues, pass-through without them: pseudo-haptics is carrying the touch, rather than a force channel the hardware already had.

Related

• Same group: N2.09.1 The contact volume must be larger than the object's visual hull · N2.09.2 The moment of release is harder to judge than the moment of grasp · N2.09.3 A mid-air press has no bottom-out signal and needs another channel
• Nearby: N2.04 Direct Grab · N2.02 Bare-hand Tracking
• Search terms: pseudo-haptics · visuo-haptic illusion · control-display ratio