Calibration requires exposing failures, not hiding them

What it is

A demo reel that only shows regex that compiled, only pretty sentences in translation, leads people to hand over as if it will always be like that. Failures hidden off the demo leave the estimate with no downward samples. Calibration depends on showing failures, not hiding them. Hiding them improves the look and cuts the match.

Showing them is not for fright. It is so the estimate can eat data from both sides.

Why it happens

Estimates of frequency lean hard on what was seen. Success is presented whole and handsome; failure gets a dialog, or nothing — what encodes in memory is success. In sparse use, a failure may be a long time coming as the user’s own collision, and by then the estimate is already high. Muir’s update from experience needs positive and negative samples; if the interface only supplies positives, the update is missing a term.

Hiding has a second form: failure rewritten as the user’s fault (“try phrasing it differently”), so the system’s miss rate is remembered as zero. The estimate is then lined up with an edited world.

Studying it

Same model: one group sees successes only; another gets typical failures inserted at the same count (including fluent errors). Then measure estimate and handover. Independent variables: whether failures are shown, whether the failure is attributed to the system or the user, form of failure (no output / fluent error). Dependent variables: calibration error, scope of handover, surprise at a later real failure.

Inserted failures must be the kind that actually occur on this task. An unrelated crash animation does not count.

Where it stops holding

Laying out every failure on first use will drive people off — that is excess, not a duty to hide. Failure display on high-stakes features needs a check people can actually take; a naked display with no next step produces helplessness. This entry says calibration needs failure samples. Using typical failures to press down already-high trust, and the fact that trust cannot be built from documentation, are a different set of operations. How one severe incident punches through trust is also not this timescale.

Applying it

• In empty states and onboarding, put at least one typical failure for this task on the same layer as a typical success; do not only put the best once.
• When a failure happens, mark that the system did not do this one well. Do not default to writing that the user asked badly.
• In settings or “about performance,” give a recent failure count, not only a success rate.
• Check: before the first handover, can a new user name one way this system gets it wrong. If they cannot, failures have been hidden; then see whether their handover scope is clearly wider than your base rate.

Related

• Same group: L5.03.1 The goal is for trust to match actual reliability · L5.03.2 Overtrust and undertrust are both failures
• Nearby: L5.09 Overtrust and Trust Collapse · L5.04 Collapse of Trust · L1.06 Graceful Degradation of AI Failure
• Search terms: expose failures · trust calibration · hidden failure