Heuristic evaluation yields an issue list, not user data
Aliases: inspection issue list · expert prediction · user evidence
What it is
Heuristic evaluation produces inspection findings: principle-based predictions about where users may be hindered, not observations of failure, prevalence, or experience. The list supports early repair and hypothesis formation but is not evidence that “users proved” a problem.
Why it happens
Evaluators map interface states to principles and infer consequences without target users performing tasks. This exposes prelaunch risks but creates false positives users bypass and misses problems dependent on real knowledge, motivation, and setting.
Studying it
Record screen evidence, trigger, violated principle, predicted consequence, and confidence. Compare against usability sessions, support incidents, or telemetry for hits, unique findings, and false positives. Nonoccurrence in a small user sample is not proof of absence; validation must match state and population.
Where it stops holding
Absence of user data does not mean no evidence: safety, accessibility, and consistency can rely on standards and engineering facts. User tests are not the sole truth either. The requirement is to label evidence type and avoid presenting predicted prevalence or severity as observed.
Applying it
- Label each item as an expert prediction with state, consequence, and uncertainty.
- Fix clear low-cost issues; validate disputed high-risk ones with users.
- Never infer affected-user proportions from issue counts.
- Reinspect the principle after repair and test practical impact in target tasks.