The same problem often fits several heuristics; classification disagreement does not invalidate the problem
Aliases: heuristic classification · classification disagreement · usability problem
What it is
One interface problem may involve status visibility, feedback, recovery, memory load, and control simultaneously. Heuristic overlap means labels are analytical tools, not the condition for a problem’s existence; as long as the situation, user goal, and consequence are described, disagreement about category does not weaken the finding.
Why it happens
Heuristics slice one interactive system from different angles. A long wizard without an exit is both control and closure; an upload without progress is feedback, memory, and waiting. Evaluator experience, attention path, and wording lead to different labels. Requiring one category makes teams argue over nouns and may discard genuine defects.
Studying it
Record each problem with label-independent fields: interface location, triggering task, expectation, actual result, scope, and evidence. Then allow multiple tags and analyze tag co-occurrence. Multi-rater annotation can study classification agreement, but distinguish “different problems” from “different classification.”
Where it stops holding
Multiple tags do not justify inflating issues. Merge duplicate reports with the same trigger and consequence, keeping one problem record; tags organize repair directions. If disagreement comes from different understanding of state or task goal, gather evidence rather than averaging opinions.
Applying it
- Use a two-layer problem template: facts—location, task, expectation, actual result, consequence—and classification—one primary plus related tags.
- Confirm facts and severity before discussing category; never delete a report because its label differs.
- Route problems using related tags to feedback, recovery, navigation, or content repair.
- Review high-co-occurrence tags periodically to decide whether to merge criteria or add product-specific checks.
Related
- Same group: B3.18.2 Heuristic entries are not mutually exclusive; counting problems per entry double-counts · B3.18.3 Too many entries reduce the proportion actually used by evaluators · B3.18.4 Problems fitting no entry should be kept; they show the current set is incomplete
- Nearby: Q2 Usability Evaluation · Q4 Research Methods and Evaluation
- Search terms:
heuristic evaluation·classification agreement·usability problem