Tracking loss is the normal path, not an error branch
Aliases: dropout as expected · hand-loss recovery · tracking gap as state
What it is
A hand goes behind the back to rest, reaches across the body, or is simply covered by a sleeve — the virtual hand vanishes and returns tens of times in a session. For bare-hand tracking, tracking loss is the normal path, not an error branch. It should not become a modal dialog, a crash replay, or a blocking “please move your hands into view.” Loss and recovery are two ordinary states on the main path.
The cause may be occlusion, facing, light, or leaving the cone; this leaf does not recatalogue causes. It claims that once loss will happen, the interaction state machine must treat “hand absent” as a legal peer of “hand present.”
Why it happens
Visual tracking is a cycle of see–estimate–emit. The unseen interval is not a fault; it is a phase the cycle always contained. Hands leave cameras by design: they drop, cross, enter pockets, reseat a headset. If the system treats that phase as exceptional, every departure triggers a recovery ritual — freeze input, show a prompt, demand a calibration pose. The ritual costs more than the loss, and people learn not to rest their hands, shoulders locking up with them.
A legal “hand absent” needs explicit semantics: is an in-flight grab cancelled, held, or handed to the other hand; does the pointer vanish, freeze, or snap to a default near the body. Vague semantics make the recovery frame the most dangerous moment — a hand that pops back into existence can complete an accidental pinch over a button. Normalising loss is not “prompt when it drops.” It is “loss and return already have written consequences.”
Studying it
Treat loss as a factor, not as noise to filter. In a complete task, insert intervals where the hand leaves the view (behind the back, covering a camera, hanging by the thigh) and watch the state machine.
Independent variables: duration of loss, whether an object was held at dropout, whether the recovering hand lands on an interactable, policy during loss (hide / freeze / snap to body). Dependent variables: accidental triggers, whether the task is interrupted, whether users report the loss as an error, accuracy of the first action after return.
The interesting quantity is not loss rate, but how users classify the loss. If interviews yield “it crashed,” the state machine turned a normal phase into an exception. Logs should record “hand left” and “hand returned” separately, tagged as rest, hand-switch, or mid-action.
Where it stops holding
Surgery, teleoperation, and other settings that cannot tolerate a hollow input frame treat loss as a true fault and need redundant sensing; a state machine cannot digest it. Controller tracking is a different pattern: LEDs or IMUs on the shell keep integrating through short camera gaps, so the bare-hand premise of “often completely unseen” does not hold. Lab protocols that keep both hands raised at the chest drive loss toward zero and produce a false “stable.” Take the device home, sit on a sofa, let a hand fall onto a cushion, and the normal path returns. Children and untrained users park hands in camera dead zones more often; treating loss as error punishes them, not the practised demonstrator.
Applying it
- Give the input state machine two peer states: hand visible, hand not visible. Entering the latter is not an error; it only changes pointer and grab semantics.
- An in-flight grab at loss needs a decided fate: release, pin at the last pose, or hand off to the still-visible hand. Do not wait until recovery to guess.
- When a hand reappears over a button, insert a short dead zone or an explicit re-entry action so the recovery frame cannot commit.
- Rest poses (hands hanging at the sides) are first-class: the interface stays usable or enters an explicit idle, rather than repeatedly ordering the hands up.
- How to check: in a full flow, require three “hands behind the back for five seconds, then back.” If none of the three produce a mis-click, an error string, or a broken task, loss is on the main path. If any one shows “tracking failed,” the state machine still treats it as an exception.