Tracking loss must be distinguished from ordinary action gaps so it is not treated as disengagement

What it is

In mid-air work the hand already pauses and changes pose between actions: that is an action gap. Tracking loss is the sensor temporarily failing to report where the hand is, even though the hand is still there. Both can look in a log like “no valid joints for a while,” with opposite intent: a gap is a pause while still engaged; a loss is a broken channel. Treating loss as a gap lets the system guess while blind; treating a gap as loss (or as leave) writes a breath as an exit. What must be split is “is the hand still in the picture,” not whether engagement grace should be kept—grace is a session-layer question.

Why it happens

During an action gap, keypoints are usually still present, velocity near zero, or following a predictable return path. During loss, keypoint confidence collapses, the skeleton vanishes or jumps to an impossible pose, and it may come back under another ID. Discriminating quantities include a steep drop in confidence, a depth hole, a spatial jump from the last frame, and whether the torso is still visible. Watching only “was a gesture issued” mixes the two, because a gap contains no gesture by definition. The typical path to a false exit is: gap is written as loss, loss is written as leave, and the user was only changing grip between pinches.

Studying it

Record two classes of clip with ground truth: loss from intentional occlusion, and scripted pauses (count to three, then continue). Classify on confidence, jump distance, and duration, and report both errors: gap labeled loss, loss labeled gap. Feed both classes into the product state machine and count how many pauses become exits. The participant task must include deliberate no-motion waits; otherwise the training set only contains “always moving” and the gap class does not exist.

Where it stops holding

High-speed motion itself lowers confidence; in dance or sport, “really moving” and “lost the track” are harder to split and thresholds must be retuned per task. A controller or glove that still reports contact or inertia can prove “the hand is still there” when optics drop, which makes the split easier. Near-field modules that see only the hand, not the torso, lack a side channel for “the person is still here” and more easily write gaps as loss. A one-frame hole is almost always noise and does not deserve promotion to a loss event.

Applying it

• Give the state machine two states: “no gesture but the hand is still tracked” and “hand tracking is broken.” Forbid a single lost boolean.
• During a gap, keep engagement and focus. During loss, enter dedicated loss handling (freeze or prompt); do not treat it as the user choosing to stop.
• Accept with two recordings: “pinch, wait two seconds, pinch again” and “occlude with a card for 300 ms mid-pinch.” The first must not enter loss; the second must not be treated as a normal pause.

Related

• Same group: C4.23.2 In-progress continuous operations should freeze rather than zero or cancel when tracking is lost · C4.23.3 After tracking recovers, the system must decide whether to resume the frozen state or require a restart · C4.23.4 Prolonged tracking loss should prompt the user to adjust position or lighting
• Adjacent: C4.19 Re-entry grace · C4.03 Gesture onset and offset detection
• Search: tracking loss · action gap · dropout