Telling users the correct procedure rarely overrides an entrenched wrong model; it takes a first-hand experience that falsifies the old one

What it is

This entry deals with a situation different from a misunderstanding that just formed: the user has already been operating on a wrong understanding for quite some time, and that understanding isn't an untested guess — it's a belief that has been used and "confirmed" by the user's own repeated experience: an entrenched wrong model. Facing a model like this, simply telling the user "the correct way is actually this" usually does little. What actually works is having the user go through a first-hand experience where the old model's own prediction gets broken by reality — a falsifying experience.

Why it happens

Behind an entrenched model sits a body of evidence the user has built up through their own repeated successful operations — evidence that is large in volume, personally lived through, and confirmed by an actual outcome every single time. A verbal statement from an outside source, however authoritative or clearly worded, is just one isolated piece of new information, nowhere near the weight of that long-accumulated, repeatedly confirmed personal experience. Put the two on a scale, and the verbal statement is almost destined to be the lighter side.

More fundamentally, once a person holds a belief that has been working for them, they naturally tend to process new information in ways that preserve that belief — a statement that doesn't fit gets treated as an exception, misread, or simply ignored, rather than as evidence that the existing belief needs to be overturned. Simply telling someone the correct answer can't route around this tendency, because telling is still just a claim waiting to be verified, and the user can shelve or reinterpret it exactly as they would any other piece of information that doesn't match their expectation. What can actually shake a belief like this is having the user personally carry out the prediction their own model generates, and then watch, with their own eyes, the result fail to match. This new evidence sits in the same evidential category as what originally built the old model — both are first-hand, lived experiences that can't be waved away as "I must have misheard" or "I misremembered" — which is what qualifies it to compete head-on with the old model, instead of being quietly absorbed by it.

This is also why the situation described here is not the same stage as timely explanation preventing a wrong model from forming in the first place: that earlier situation is about the window right after a violation happens, right as a gap opens, before the user has had time to confirm a belief through their own repeated use. An entrenched wrong model is exactly what shows up after that window has already closed — after the user has already confirmed some account many times over through their own experience. What works during that earlier window — stating the cause clearly, and stating it promptly — is aimed at a gap that is still empty and waiting to be filled. What has to be dealt with here is a belief that has already been filled in, with contents that have already been repeatedly verified; the gap is long gone, and trying to fill a gap that no longer exists with one more explanation naturally has nothing to work on.

Studying it

This corresponds to the refutation text paradigm from science-education psychology: a text first explicitly states a common misconception, then specifically explains why it's wrong, and finally presents the correct understanding — this structure is compared against an ordinary explanatory text that only states the correct information and never mentions the misconception, measuring how well each clears the misconception on a delayed test.

Common independent variables: whether the text explicitly names and refutes the common misconception, the interval before the delayed test. Common dependent variables: recurrence rate of the misconception after the delay, retention of the correct concept, change in the participant's confidence in their original belief.

Transplanting this paradigm to interface contexts calls for caution: most of the underlying research involves students reading course material in a formal learning context, actively cooperating with the expectation that "this is studying." Product users rarely read a passage of explanatory text with that same mindset. The clearing effect measured in a lab setting shouldn't be assumed to reproduce itself in real use — what actually tends to work is a design that lets the user personally run into a falsifying result, not the text itself.

Where it stops holding

• A model only counts as "entrenched" once it has actually been confirmed through multiple successful uses. A single attempt with an ambiguous outcome, not yet repeatedly verified, is outside the scope of this entry — that kind of misunderstanding is at an earlier stage, where a timely, well-anchored explanation can still work.
• How entrenched a model is scales roughly with how often and how long it's been used — not every wrong belief starts from the same point. Deciding whether telling or a falsifying experience is called for requires first judging how many times this particular model has already been confirmed.
• This entry only covers why a falsifying experience is necessary — it does not cover how large that experience should be or how it should be delivered. The scale and delivery of the experience affect whether it's actually accepted by the user, and that's a separate matter.

Applying it

• Mine support records or recurring error logs for misunderstandings that have "already been explained, yet keep coming back" — recurrence itself is the most direct evidence that a model has become entrenched and that telling alone won't work, with no need to guess.
• For misunderstandings confirmed to be entrenched this way, stop investing further effort in polishing the explanatory copy, and instead design an interaction moment where the user carries out an action driven by their old model and can clearly see the result fail to match their expectation.
• How to check: after introducing a falsifying experience, watch whether the wrong behavior stops showing up in later, independent usage sessions — not just whether it gets corrected on the spot within that one session. A genuine correction should hold up without any prompt; an in-the-moment "oh, I get it now" during a single session isn't enough to prove the model has actually been rewritten.

Related

• Same group: A7.14.2 Correction should start with a small-scale, controlled-consequence falsifying experience, not a one-shot fix · A7.14.3 A correction delivered too bluntly triggers resistance and can strengthen the original wrong model · A7.14.4 A transitional analogy between the wrong model and the correct one reduces the cognitive leap required to correct it
• Nearby: A7.11 Expectation violation and the explanation gap · A7.10 Making the conceptual model explicit
• Search terms: refutation text · belief perseverance · conceptual change · falsifying experience