Threshold setting is a product decision

What it is

The balance point between false alarms and misses has no technically correct answer: on the same hardware, tuning the threshold twitchy or sluggish yields two equally feasible product stances — "safety first" or "quiet first", "rather ask once more" or "don't bother me". Calling the threshold "decided by the algorithm" hides a value judgement behind technical vocabulary. The threshold is policy.

This is not a rhetorical observation: the default threshold determines the user's first impression and trust baseline for the function, and thereby whether it stays on or gets switched off. It is product behaviour and belongs under product decision processes and accountability.

Why it happens

Setting a threshold is choosing an operating point on the ROC curve — any point is technically achievable; what differs is the allocation of consequences: who gets disturbed, who carries the risk of the miss. Three mechanisms make the default matter far more than it appears:

• The default is the baseline: most users never touch settings, so the factory threshold is policy for life — not a suggestion but an execution.
• First impressions write trust: a first week dense with false alarms and the user switches the function off, permanently; weeks of silence and the user distrusts it when a real event comes. The operating point writes itself directly into the user's mental model of the system.
• Liability and compliance enter the threshold: smoke and carbon-monoxide alarms follow mandatory standards; child monitoring and fall detection sit in a void — within one product line, the discretionary part is exactly the part with the most ambiguous consequences.

"The algorithm decides" is popular because it spares both sides: engineering skips justifying the consequence allocation, product skips signing off on alarm frequency. The cost returns at failure time as "why does the product behave like this" — owned by no one.

Studying it

• Perception of operating points: interview studies of smart home and alarm systems consistently find users cleanly distinguish "too sensitive" from "unreliable" complaints and act accordingly (switch off vs. stop trusting) — evidence that the operating point reads as product attitude, not technical detail.
• Evolution of alarm-management standards: clinical alarm management moved from "more alarms, more safety" to tiering and reduction through a traceable standards process — operating-point adjustment became institutionalised governance, a reference other domains can borrow.
• A/B and longitudinal tracking: controlled comparisons of default thresholds on function retention, alert dismissal, and miss costs directly measure an operating point's consequences.

Methodological caution: short laboratory evaluations inherently favour sensitive settings (participants are told to hunt for events and tolerate false alarms); an operating point's real consequences unfold over weeks of home use — the optimal threshold in week one and week ten often differ.

Where it stops holding

• Not every threshold is free to set. Life-safety functions under mandatory standards (smoke, CO) follow the standard; discretion exists only in the grey zones standards do not cover.
• "Let users adjust everything" is not the answer. Adjustment addresses between-household variance (a sibling topic); it does not dilute the default's importance — most users never adjust, and the default remains the dominant operating point.
• Thresholds are not set once. Software updates move models and thresholds, silently changing the product stance without the user's knowledge — threshold changes deserve review at the same level as initial setting.

Applying it

• Write a one-sentence product stance statement per detection function ("rather ask once more than miss" / "quiet first, misses are on the user"), into the requirements document and through team review — so the value choice has an owner on record.
• Calibrate the default for the most vulnerable user: a lone older adult and student flatmates have different optimal points; the product positioning decides which one — and if you cannot say which, the positioning is not yet decided.
• Route changes to thresholds and detection models through product review, not pure algorithm tuning; tell users explicitly when an update changed sensitivity.
• How to check: after a new default ships, track three movements — alert dismissal rate, function retention, and drilled miss rate. Rising dismissal = too twitchy; falling retention plus "unreliable" complaints = too sluggish. Both directions need someone watching the board.

Related

• Same group: Z2.03.1 The costs of the two error types are usually asymmetric · Z2.03.3 Users need adjustable sensitivity
• Nearby: Z2.02.3 High-stakes actions should not fire on a single inference · Z3.01 Levels of initiative
• Search terms: operating point · ROC analysis · alarm management · default settings