A model can be incorrect and still support successful operation

What it is

A user's understanding of a system can be entirely wrong — believing that a deleted file "is still sitting somewhere, just hidden" — and yet they get through their daily tasks just fine, as long as that understanding never generates a mispredicted outcome within the range of operations they actually touch. Usability doesn't depend on whether a model is objectively correct; it depends on whether the model's predictions happen to match the system's real behavior across the slice of the operation space the user actually visits.

Why it happens

Any given user only ever touches a small subset of a system's full behavioral space — the common functions, the familiar paths. As long as an incorrect model happens to agree with the system's real behavior across that subset, the error never surfaces, because exposing a wrong model requires a concrete scenario where the model's predicted outcome and the system's actual outcome noticeably diverge. If the user never reaches such a scenario, that divergence simply never happens, and the model keeps looking correct indefinitely. This means how long a wrong model can hold up is really a function of how much the user's actual operating range overlaps with the system's true behavior, not a function of how many errors the model contains.

Studying it

Testing whether a model is merely "accidentally right in common scenarios" requires prediction tasks deliberately designed to push users into atypical, low-frequency situations, then checking whether the model's predictions still hold there. Simply tallying error rates during everyday use can't answer this question, because the error rate itself is set by how much the model's coverage overlaps with actual usage — more overlap means fewer errors regardless of whether the underlying model is correct.

Where it stops holding

Whether a model keeps holding up depends on whether the overlap between the system's behavioral space and the user's actual operating range stays stable — once the system updates, or the user for the first time steps onto a path they've never taken before, that overlap breaks and the dormant error surfaces immediately, producing a jarring mismatch with expectation. It's also worth separating "works" from "works well": even a wrong model that never triggers a visible failure can quietly send a user the long way around, or lead them to give up on features that exist but don't fit their mistaken understanding — a hidden cost that never shows up in an error rate and that the user themselves usually never notices.

Related

• Same group: A7.01.1 A mental model is a user's internal explanation of how a system works · A7.01.3 The model determines a user's expectations and response when something goes wrong · A7.01.4 The core function of a mental model is predicting system behavior for a given action, not memorizing steps · A7.01.5 A sufficiently complete mental model lets a user derive operations they were never taught · A7.01.6 A model's internal consistency and its accuracy are two separate things — a consistent but wrong model still works · A7.01.7 Users lack insight into the accuracy of their own model and rarely question it
• Nearby: A7.14 Identifying and correcting a wrong mental model · A7.06 Identifying a flawed model
• Search terms: working misconception · folk theory accuracy · model coverage