Hand occlusion is the error people notice most
Aliases: near-field hand occlusion · hand depth · self-occlusion of the hand
What it is
Reach for a virtual cup and the fingers should hide the rim. If the rim hides the fingers, or the whole cup leaks through the gaps, the error is almost never filed with “that distant lamp is also a bit off.” Hand occlusion is the mixed-reality occlusion error people notice most: the hand is always near, always in the gaze, always the patch proprioception has already declared “I am here.”
This is not a generic plea for correct occlusion. It is that once the hand–virtual order flips, people appraise their own hand before they appraise the room.
Why it happens
The hand emits three signals that check each other: the retinal image, the felt position, and the rule that a hand should hide whatever is farther. A virtual object that inserts itself among those three lands in the fovea, at short range, where disparity is largest. Depth sensors are especially bad here: fingers are thin, motion is fast, skin is unstable in infrared, and the mesh cannot keep up with opening gaps. If a skeleton stands in for depth, the volume misses the flesh, and virtual surfaces still leak through the seams.
Attention leans toward the hand as well. Grab, press, and pinch put it in the task focus. Occlusion errors on peripheral furniture can be missed by not looking; the hand has no “look less” escape.
Studying it
A near-field grasp: park a virtual object where the real hand must occlude it from the front. Record, without prompting, whether people mention the hand, and whether gaze parks on the wrong frame. A control parks the same class of error on furniture in a corner and compares spontaneous report rate. Body-ownership and rubber-hand setups can help, but the dependent measure should be “did they notice the hand,” not “did it feel like their own.”
Independent variables: depth order of hand versus virtual object, hand-mesh source (depth / skeleton / none), movement speed. Dependent variables: unprompted report rate, time to first fixation on the error, whether the grasp still completes.
Spontaneous comments mid-task are cleaner than a post-hoc questionnaire — hand errors are often spoken before anyone asks.
Where it stops holding
With controllers, the real hand is often out of view or hidden by the hardware; the error migrates to controller-model versus virtual object. The mechanism stays, the object changes. Far-field, no-reach interfaces (distant panels, gaze-and-pinch) take the hand out of focus and demote hand occlusion. Opaque gloves or fiducials make hand depth easier to measure, and the “noticed most” status drops in the lab. If a self-avatar replaces the hand with stylized geometry, people no longer expect real skin; the reading shifts from “my hand went through” to “this mesh is misaligned.”
Applying it
- For near direct manipulation, treat the real hand as its own depth layer. Use a hand mesh if you have one; do not carve fingers with the room-scale scene mesh.
- When depth cannot keep up, pull the virtual object back as a whole or fade it near the hand. Do not paint a rim on a fingertip.
- For grabs and presses that must put the hand in frame, make “hand in front” a pass condition, not a visual patch after launch.
- How to check: ask people to pick up a virtual cup on the table and do not mention occlusion. On replay, count how many named the hand without being asked. That rate should beat comments on chair legs and corners if “noticed most” is true; if everyone is talking about something else, the hand is not the loudest error in this build.