The costs of the two error types are usually asymmetric
Aliases: false positive / false negative · Type I / Type II error
What it is
Every detection system makes two kinds of error: false positives (no event, but one reported) and false negatives (an event, missed). Under a single threshold, pushing one rate down necessarily pushes the other up — a mathematical relation with no two-ends-low setting. The essential insight for design: the two costs are almost never symmetric. A missed intrusion alarm is an unnoticed break-in; a false one is a walk down the hall. A missed smoke alarm is a fire; a false one is a fright. The first task in designing a detection function is not tuning parameters but answering: which error is more expensive here?
Treating "maximise detection rate" as the objective implicitly declares misses infinitely costly and false alarms free — neither holds.
Why it happens
Three sources of asymmetry:
- Magnitude differs: what gets missed is usually safety and opportunity (intrusion, falls, leaks); what gets falsely raised is mostly convenience (one needless check) — though the reverse exists too (social embarrassment, waking the household at 3 a.m.).
- The bearer differs: missed events are paid by the user; false alarms are paid by the product's reputation ("this thing keeps going off") — so product decisions drift systematically toward suppressing false alarms, even against the user's interest.
- Reparability differs: a false alarm is clarified on the spot ("nothing, just a glitch"); a missed event is gone for good — irreversibility in time inflates the per-instance cost of misses.
User adaptation amplifies the initial asymmetry: repeated false alarms train users to ignore alerts — the cry-wolf effect, extensively documented as alarm fatigue in nursing and aviation. At that point the nominal detection system delivers near-zero protection: a threshold pushed hard against false alarms ultimately buys systematic misses.
Studying it
- Signal detection theory: separates detection into sensitivity (d′, true capability) and response bias (β / criterion, the operating point) — the standard framework from Green and Swets' classic work. It establishes the point that matters most for design: threshold tuning moves β, not d′ — complaints of "inaccurate" and complaints of "too twitchy" require different diagnoses.
- ROC analysis: plot both error rates across the full threshold range and inspect the curve shape before choosing an operating point — the area under the curve tells you how finely this sensor is worth tuning at all.
- Alarm fatigue research: the clinical alarm-management literature quantifies the relation between alarm volume and response rate (response declines monotonically as volume rises), evidencing that "the true cost of a false alarm includes future misses".
Methodological caution: misses are unobservable in natural use — the user never knows an unraised event happened; field measurement requires injected events or post-hoc annotation. An evaluation counting only false alarms systematically overestimates the system.
Where it stops holding
- The cost ratio drifts with situation and time. A false alarm is cheap by day (someone is home) and dear at night (waking everyone); an "average false-alarm cost" hides this distribution — time-of-day thresholds are the right granularity.
- Subjective complaints are not an unbiased cost estimate. False alarms are over-represented — experienced and remembered; misses are under-represented — silent and unfelt. Feeding complaints straight into the cost ratio tilts systematically toward suppressing false alarms.
- For "both errors fatal" scenarios no tuning helps. The problem there is sensitivity, not bias — change modality or add fusion; the ROC curve itself must move.
Applying it
- For every detection function, write a two-line cost statement — "what happens on a miss / what happens on a false alarm" — in concrete consequences, and set the threshold direction from it. This is a product decision, not a parameter tweak.
- Monitor the actual rates of both errors after launch plus the user's alert-dismissal rate; a rising dismissal rate is the leading indicator of alarm fatigue.
- Where costs vary by time of day, use time-segmented thresholds: nights lean toward not disturbing — except for safety functions (smoke, falls), where the reverse holds: night is precisely when misses cost most.
- How to check: tally both error rates after the fact — false alarms from user feedback, misses from injected-event drills — and compare against the design's cost assumptions. Both numbers must be watched; watching only false alarms amounts to declaring misses free.