Recognition systems report a confidence for each decision rather than a binary outcome

What it is

A gesture recognizer does not flip a “yes / no” coin on an observation. It issues a soft decision with a confidence: more like this class, less like that one, and how sure overall. The product layer then cuts the score into a command. Treating the recognizer as a black-and-white switch throws away uncertainty the score already computed. Tracking loss is “no observation”; low confidence is “an observation, but unstable.” Do not merge them into one “not recognized.”

Why it happens

Classifiers emit a posterior, a distance, or a calibrated probability. The same trajectory can give mid-range scores to several classes, or a high class score with low keypoint quality. Binarization happens after a threshold; before the threshold the information can still delay a trigger, demand confirmation, or fall back to a stabler template. Calibration matters: many models’ raw scores cannot be read as probabilities, and 0.8 does not mean “eight times out of ten.” Showing uncalibrated scores to users manufactures false surety. Frame-level scores and scores over a whole gesture span are also different; a one-frame peak does not mean the whole motion was reliable.

Studying it

On a labeled vocabulary, plot score distributions for the true class, confusable classes, and non-gesture motion. Check whether scores move monotonically with keypoint quality, speed, and viewpoint. Use a reliability diagram to see whether scores are calibrated. Compare product logic that uses only argmax with logic that hands scores downstream, on the same test set. Do not report only top-1 accuracy; that number is exactly the soft output after it has been crushed to black and white.

Where it stops holding

On a toy vocabulary with few templates and huge between-class distance, scores pile at the ends, the mid-range almost vanishes, and binarization loses little. In the open world most motion is “none of the above,” and mid-range scores are the normal case. Some sensors only offer a detection switch (a simple proximity threshold) with no interpretable confidence; this claim does not apply, and a switch voltage must not be renamed confidence. Temporal models’ scores swing hard before the motion ends; a mid-gesture number is not end-state surety.

Applying it

• Have the recognition service return class, score, and keypoint quality together. Forbid the product layer from subscribing only to a boolean recognized.
• Show scores in the debug view. Do not present uncalibrated percentages to users as a “success rate.”
• At acceptance, sample “correct but low score” and “wrong but high score” cases and confirm the downstream can still see them, rather than losing them in a binary interface.

Related

• Same group: C4.24.2 At low confidence the system must choose whether to prefer misses or false positives · C4.24.3 Bias direction should follow consequence severity rather than a uniform setting · C4.24.4 Adjusting the confidence threshold is a product decision, not something to inherit from the model default
• Adjacent: C4.23 Handling tracking loss · C4.03 Gesture onset and offset detection
• Search: confidence · soft decision · score calibration