High-stakes settings must not leave checking entirely to the user

What it is

A dose suggestion finishes generating on a night shift, with “please check the package insert” beside it. The person checking is already watching three monitors; there is no second, maintained deterministic path. Once harm happens, the system treats “you didn’t check” as a completed allocation. Non-delegable checking means that when the cost of error exceeds the attention the user can spend on the spot, verification must be carried by system-side constraints, checks, or refusal — not assigned as homework.

A disclaimer writes responsibility on paper; that is a different object. This entry governs whether checking labour is actually performed before the incident.

Why it happens

In high-stakes work, the attention budget is already spent on the situation: the patient, the amount, the legal deadline. Generation stacks a new checking task whose difficulty is inverse to the generator’s fluency. “Please verify” completes a procedural handoff without supplying the time window, the comparison object, or a halt when checking fails.

If allocation exists only in copy, actual behaviour follows least effort: output that looks normal is adopted. High stakes plus “looks normal” means offloading the check leaves incident probability with the person on scene.

Studying it

In tasks with real (simulated) cost — medication, a transfer, a legal filing — compare three: disclaimer only, disclaimer plus a checklist, system-side hard checks (dose range, account allow-list, clause template). Dependent variables: rate of unchecked adoption, number of errors caught, time to complete. Independent variables: time pressure, whether the check interrupts the main path.

“Participants said afterwards they would check” is not evidence. Watch whether the comparison object was opened before submit. A time-pressure condition is required, or the lab will overestimate checking.

Where it stops holding

Low-stakes reversible work (a draft title, internal brainstorming) can leave checking with the user. When the user is a professional checker in that domain (an editor, a pharmacist at their own bench) and the system supplies a comparison object, checking can be part of the job — still not a lone “please verify.” Where the system cannot check, the right degradation is refusing factual output, not generating and offloading. This entry does not treat what a footer does to spread. It treats whether checking is performed before harm.

Applying it

• For dose, money, legal consequence, identity: run constraints and checks first; on failure, do not emit a copyable fact sentence.
• If a human check is still required, it must interrupt the main path: comparison object side by side, a confirm control before submit, no one-click adopt as default.
• Do not treat “please verify yourself” as the complete design for high stakes. It supplies neither time nor a comparison object.
• Check: run the target task under time pressure and count how many people opened the comparison object before submit. Near zero, with errors still submittable, means checking was offloaded and not performed.

Related

• Same group: L3.03.1 Fluent wording is not the same as being correct · L3.03.2 The cost of checking can exceed doing the work oneself · L3.03.4 When errors sit inside correct content, checking is sentence by sentence and costs about as much as rewriting · L3.03.5 The less familiar the domain, the harder checking is — and that is exactly when people ask the system · L3.03.6 Certainty of wording has no relation to reliability of content · L3.03.7 Errors in numbers, dates, and names are the hardest to notice and the most damaging · L3.03.8 Putting the checking duty in a disclaimer does not reduce how far errors actually spread
• Nearby: L1.05 Human in the Loop · L4.11 Pre-action Confirmation and Irreversible Operations
• Search terms: non-delegable checking · high-stakes verification · offloaded fact-check