Correction should start with a small-scale, controlled-consequence falsifying experience, not a one-shot fix

What it is

Once it's established that correcting an entrenched wrong model requires a falsifying experience, the next question is how big that experience should be. The answer isn't "as decisive as possible" — correction should start with a small-scale, consequence-controlled falsifying scenario, letting the user run into a failed prediction at low cost, rather than handing them a single, high-stakes humiliation inside a real, consequential task.

Why it happens

A falsifying experience that carries real stakes tends to pull the user's attention away from "my understanding needs updating" and toward "how do I deal with this loss." High stakes on their own trigger a defensive response — the user is more likely to chalk the failure up to bad luck, a temporary glitch, or some other external cause than to admit their long-held understanding was wrong. The higher the stakes, the more motivated the user is to find an account that protects the existing belief, which cancels out exactly what the falsifying experience was supposed to achieve.

A small-scale, consequence-controlled scenario removes this interference: the user can focus entirely on "the result wasn't what I expected," without also having to manage the emotional reaction that comes with a real loss, and it's easier for them to treat the failed prediction simply as new information about how the system works, rather than as an incident requiring defense or damage control. Controlling the stakes has another benefit too: even if the first falsifying experience doesn't fully convince the user, low stakes mean it can be repeated — building up evidence gradually through repeated exposure, rather than betting everything on one irreversible moment.

Where it stops holding

• The scenario can't be so small that the user doesn't notice anything is off — a falsifying experience needs to be salient enough to register as "this isn't what I expected." Too mild a discrepancy risks being ignored or rationalized away, failing to break the old model — a risk on the opposite end from stakes being too high, and just as capable of derailing correction.
• The right scale isn't a fixed value; it depends on how entrenched the original model is. A model that's been confirmed many times, held with high confidence, may need a more noticeable mismatch than a fresh mistake would to even be perceived — "smaller is always better" doesn't hold either.
• This entry is about how to set the stakes and scale of a falsifying experience — it doesn't cover the tone or wording used to deliver it. How bluntly or gently the result is communicated is a separate layer, and it independently affects whether the user is willing to accept the failed prediction.

Applying it

• For a wrong model already confirmed to be entrenched, design a specific low-stakes scenario — a sandboxed practice moment, a reversible action, a preview flagged in advance as "just a demonstration" — where the user acts on the old model's prediction and then sees an actual result that doesn't match, while making sure that mismatch carries no real cost.
• Avoid placing the first falsifying experience inside a formal, consequential task flow. If correction can only happen within a real task, accept that cost for now while logging it as evidence to justify building a low-stakes scenario going forward.
• How to check: after falsification happens in the low-stakes scenario, track whether the user still repeats the old-model-driven behavior in real, consequential usage. If the correction only holds inside the demonstration scenario and reverts the moment the user is back in a real task, the low-stakes scenario wasn't designed close enough to the actual decision context, and needs to be brought closer to it.

Related

• Same group: A7.14.1 Telling users the correct procedure rarely overrides an entrenched wrong model; it takes a first-hand experience that falsifies the old one · A7.14.3 A correction delivered too bluntly triggers resistance and can strengthen the original wrong model · A7.14.4 A transitional analogy between the wrong model and the correct one reduces the cognitive leap required to correct it
• Nearby: A7.11 Expectation violation and the explanation gap · B2.12 Error tolerance and reversibility
• Search terms: low-stakes falsification · sandbox correction · belief revision · conceptual change