Occlusion by obstacles or other bodies interrupts continuous motion tracking

What it is

A cup, a screen bezel, the other hand, or another person's body between camera and hand cuts optical tracking. Occlusion interrupts tracking of continuous motion; it is not the classifier suddenly failing to know the pose. Furniture and crowds are occlusion sources the venue will repeat. How to freeze, how much grace, are policies after loss. What is required here is that occlusion, as an environmental condition, slices an in-flight trajectory.

Why it happens

A monocular or single-depth camera integrates along a line of sight; an opaque object in front eats the pixels behind. Self-occlusion is dense in mid-air: turning the palm, two-hand work, passing an object in front of the body. Other-person occlusion lasts as long as the walk-by; furniture occlusion lasts as long as someone stands behind a column. Continuous actions (scrub, held pinch) need temporally adjacent keypoints; a missing stretch either stops the integrator or lets bad extrapolation carry the value away. A discrete tap can occasionally “hit” in a gap; a continuous task cannot. Extra cameras reduce dead zones and introduce seam boundaries, another kind of occlusion edge. Transparent or reflective obstacles (glass, a screen) create false surfaces, harder to debug than a full block.

Studying it

Insert repeatable occlusions into a continuous scrub: a card, a walking confederate, the operator's non-dominant hand. Measure break length, value drift, task failure. Compare one camera with two, and with versus without predictive extrapolation. Put furniture as in the real venue into the lab and have people finish the task behind a column, not only in front of an empty desk. Design self-occlusion from palm-turns that the vocabulary actually uses, not only a front-facing open hand.

Where it stops holding

Wearable inertia or a glove may still report relative motion when optics occlude; an optical break is not a dead channel. Through-wall radar can see through some occluders; precision and finger detail are usually not enough for fine continuous work. A fully open empty room occludes little, and “continuous tracking is stable” will flip once a table is placed. Child height turns an adult-height table into full-body occlusion. What to do after tracking loss cannot be inferred from “occlusion will happen”; that is another set of decisions.

Applying it

• On the floor plan, mark where camera lines of sight are cut by tables, columns, and traffic. Do not let the interaction volume straddle those cuts.
• Avoid vocabulary that can only be completed under self-occlusion (recognizing finger pose while the back of the hand fully covers the fingers).
• Accept continuous scrubs among real furniture and aisles, not in an empty sound stage. A trajectory that fragments at an occlusion is an environment problem; change the install before the model.

Related

• Same group: C4.31.1 Reliability of vision-based gesture recognition varies with ambient light intensity and direction · C4.31.2 Strong backlight or overly dark environments significantly reduce skeleton and hand-keypoint accuracy · C4.31.4 Environmental effects should be tested with the target scene's real lighting before deployment, not laboratory ideal lighting
• Adjacent: C4.23 Handling tracking loss · C4.19 Re-entry grace
• Search: occlusion · self-occlusion · line of sight