Silent failure is more harmful than explicit failure

What it is

Generation did not do what it was supposed to do, and the interface looks as if it did: a blank is filled with a fluent “I cannot be sure, but…,” or retrieval is down and a passage that looks like an answer is still handed over. People act on success. Silent failure is worse than red type because it switches off the degradation that failure is supposed to open.

Explicit failure at least starts “what now.” Silent failure starts trust.

Why it happens

Detecting failure needs a signal other than “looks like output”: empty, schema fail, tool error, low self-consistency. If the product treats any token stream as success, the detector never rings. Generative models are unusually good at talking past a tool failure, patching the hole with narrative. Narrative satisfies “there are words on the screen,” not “the task was done.”

People inspect the interface with a completion schema: body present, no error bar, therefore done. Later acts (forward, execute, cite) fire on that schema. Harm happens upstream of degradation — the mechanism was never called.

Studying it

Three endings: an explicit error with a retreat, a silent plausible output, empty. The task has outbound send or citation. Dependent variables: rate of using the error as success, time to detection, whether a retreat is reached. Independent variables: a checker or not, whether generation is forbidden to continue on failure, whether the tool’s raw error is kept.

User satisfaction will favour the silent arm because there is no ugly red bar. The primary endpoint must be misuse of the error.

Where it stops holding

A rhetorical “I am not sure” that also blocks action (no one-click send, cannot mark complete) is not silent failure; it is uncertainty shown. Partial output with the unfinished spans marked is not silent either. True silence is success appearance plus failure fact. Pipelines whose results are independently checked even when a tool sometimes times out have a short silent window. This entry does not treat “wrong content that looks entirely normal,” where no checker ever wakes — that is a content-level false success, a separate card.

Applying it

• On tool failure, empty response, or illegal schema, do not let the model patch the hole into an article. Go to the failure panel.
• Light success appearance only after checks pass: structure gets a schema check, citations get opened, actions get a sandbox dry-run.
• Failure copy must say it did not succeed, not “here is how I understood you.”
• Check: unplug retrieval or a tool and watch the UI. If a passage that looks like an answer still appears with no error state, silence is in production. Then see whether Copy/Send still work on that passage — if they do, the harm path is live.

Related

• Same group: L1.06.1 On failure, fall back to a deterministic path · L1.06.2 The degradation order must be defined in advance · L1.06.4 Empty, wrong, and partial output need different fallbacks · L1.06.5 Fluent wrong output is the most dangerous; it trips no degradation · L1.06.6 A deterministic fallback must be maintained in peacetime · L1.06.7 Degradation must keep what the user already entered · L1.06.8 Timeouts and rate limits are expected failures and need different copy from model errors
• Nearby: L3.03 Hallucination and the fact-checking burden · I3.01 Visibility of system status · L4.13 Agent failure reporting and escalation
• Search terms: silent failure · silent degradation · false success appearance