Y2.09.2Trust cost of false alarmsdesign

A false alarm costs more than the wasted check — it also erodes trust in every alarm after it

Aliases: trust cost of false alarms · alarm trust · trust calibration

What it is

The cost of a false alarm is not limited to the visible, immediate costs of interruption, a field check, or even an unplanned shutdown — it also includes an ongoing erosion of trust in the whole alarm channel, a miscalibration of trust itself. This leaf is about what categories of cost a false alarm actually carries and which is the more expensive one; how the desensitization mechanism itself forms, how to engineer the rate down, and when the harm becomes visible belong to the other three leaves in this group. Once a given alarm source has lacked predictive value for long enough, operators lower its weight, and that adjustment carries forward onto a later genuine alarm, which then needs extra corroborating evidence before the operator is willing to act on it.

Why it happens

Trust is not a fixed property but a dynamic estimate operators continuously form about a source's reliability, intelligibility, and intent. One especially costly false alarm can damage trust more than ten minor ones combined, because the magnitude of the cost is itself a cue used to judge whether the source deserves trust. Making things worse, if an engineering fix has already corrected a false-alarm source but the system never explicitly communicates that change to the operator, the old, low-trust experience simply persists — the operator has no reason to know anything changed and keeps relying on outdated experience.

Where it stops holding

More trust is not automatically better; what should be pursued is calibrated trust, meaning the level of trust matches the source's actual reliability. Maintaining a habit of verification toward a detector that is itself still somewhat uncertain can be the correct strategy, and should not be treated as a deficiency to be eliminated. Attitude ratings gathered through a survey or interview also frequently diverge from an operator's actual response behavior in real situations, so attitude scores cannot substitute for observed behavior when drawing conclusions.

Applying it

When evaluating a false alarm, record how long it interrupted work, what checking action it triggered, what production impact it had, and whether it affected the operator's later response to genuine alarms — not just the count of false alarms. After an engineering fix to a false-alarm source, proactively communicate the logic change and validation evidence to operators rather than quietly making the change and moving on. Validate calibration using sequences that mix genuine and false alarms, checking whether the strength of an operator's response actually tracks the source's current real reliability.

Related

  • Same group: Y2.09.1 Alarm desensitization · Y2.09.3 Engineering false-alarm reduction · Y2.09.4 Latent consequences of alarm fatigue
  • Nearby: Y2.05 Alarm suppression and shelving · Y1.06 Detecting and highlighting anomalies
  • Search terms: trust calibration · false alarm cost · automation trust

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/Y2.09.2