Every mid-air gesture needs a decidable start and a decidable end

What it is

Each mid-air gesture is a token that has to be cut out, with a decidable onset and offset. Onset answers “from which frame does this motion count as a command.” Offset answers “at which frame may the system commit a result.” Without onset, the matcher does not know when to start. Without offset, the result cannot be committed, or the system hangs in “might still be going.” These are not the same edges as session engagement and disengagement. Engagement decides whether anything is being listened to; onset and offset decide where this one command is sliced.

Why it happens

Kinematically a gesture often splits into preparation, stroke, and retraction. A recognizer that watches only stroke shape will merge the raising hand of preparation or the dropping hand of retraction into the command, or commit before the stroke is done. Decidable means both edges are written in observables: speed rising from a valley, a pose crossing a shape, a hand crossing a plane—not “it looks finished.” Offset also carries commit: the class becomes a system event only at that instant. A recognizer with no offset condition can only hand in the paper on an internal timeout, so fast moves get chopped and slow ones get held. Touch gets both edges from contact; mid-air has to write them into the gesture definition itself.

Studying it

On continuous video with time stamps, have people mark the frames they consider start and end, then compare those edges with the recognizer's. Measure temporal edge error, early-commit rate, and late-commit rate, not only class accuracy. Independent variables include the onset/offset cue (speed, pose, spatial plane) and movement speed. Have the same person go deliberately slow and deliberately fast and see whether the edges follow. In Wizard-of-Oz, a human key-press as commit measures an ideal offset, not the algorithm's.

Where it stops holding

Short, high-contrast moves (a sudden fist then open) already have steep edges and can look as if offset need not be designed. Mix in a slow circle or a continuous adjustment and the hang appears. A headset that uses pinch as the edge of every hit hands onset/offset to fingertip contact; the problem moves to occlusion, not slicing. If onset and offset reuse the same action as session entry, people cannot issue a command without signing in again—the two edge pairs are welded and must be split. Children and fatigued users move rounder and slower; young-adult laboratory thresholds will miss their onsets.

Applying it

• Write an onset cue and an offset cue for every vocabulary item, and commit the system event only at offset.
• Record fast, slow, and medium productions and measure early and late commits, not only whether the class is right.
• Do not replace a written offset condition with “the recognizer will know when it is done.” In acceptance, ask operators to point at the start frame and the end frame.

Related

• Same group: C4.03.2 Return motion is part of the gesture definition · C4.03.3 Using dwell as an onset/offset signal costs waiting · C4.03.4 Onset/offset failure appears as swallowed or repeated actions
• Adjacent: C4.01 Definition and fit of mid-air gestures · C4.15 Conditions for entering engagement
• Search: gesture onset · gesture offset · stroke segmentation