Adjusting the confidence threshold is a product decision, not something to inherit from the model default

What it is

A recognition model ships a default cut with its weights: a number from a training script or a contest metric, not the operating point the product should use. Threshold adjustment is a product decision: who uses it where, what one error costs, whether retry is possible, all outrank “max F1 on the val set.” Shipping model.conf_th = 0.5 unchanged outsources product policy to whichever metric the trainer happened to use.

Why it happens

The cross-entropy, accuracy, or macro-F1 used in training weights the two errors equally, or by class frequency in the dataset, which rarely matches the live ratio of “goodbye wave” to “confirm delete.” A default 0.5 often comes from an uncalibrated sigmoid and is not even a probability. Lens height, crowd, and lighting on site shift the whole score distribution, so the lab optimum is biased in the gallery. The product may also need different operating points by country, venue, or an accessibility mode. None of that can be read from a checkpoint file. The recognition service may offer a suggested threshold; the decision should stay in product configuration, with a version that can be rolled back.

Studying it

Freeze model weights and sweep only the threshold. On a lab set, on field video, and on a set with extra non-gesture background, find “metric-optimal” and “user-unacceptable” points separately. They usually do not coincide. Run a contrast of “keep the default” against “retune on field data.” Record the decision as: who approved this number, on which batch, and when it will be re-evaluated. The method itself is the proof that the threshold is not an accessory of the model.

Where it stops holding

If the model already emits field-calibrated probabilities and the product has one low-consequence command, the default may happen to work—and should still be written as an explicit setting, so the next model upgrade cannot overwrite it with a new 0.5. Automatic thresholding (drifting with recent false-fire rate) turns the decision into an online algorithm and needs bounds, or one unusual session locks sensitivity. Thresholds reported in a contest or a paper are not field numbers; they live on another distribution.

Applying it

• Keep per-command thresholds in product configuration, versioned apart from the model file. On a model upgrade, default to “keep the product threshold”; retune only when explicitly scheduled.
• When changing a threshold, replay a set that includes background motion and inspect misses and false accepts separately, rather than the accuracy on the model card.
• Document who approved the current operating point and which data batch it used, so the next iteration does not treat “0.5” as something the recognizer requires.

Related

• Same group: C4.24.1 Recognition systems report a confidence for each decision rather than a binary outcome · C4.24.2 At low confidence the system must choose whether to prefer misses or false positives · C4.24.3 Bias direction should follow consequence severity rather than a uniform setting
• Adjacent: C4.25 Predefined fallback paths · C4.05 Three dimensions of gesture evaluation
• Search: threshold · operating point · product policy