Heuristic evaluators inspect independently before aggregation
Aliases: heuristic evaluation · independent inspection · issue consolidation
What it is
Heuristic evaluation commonly has multiple evaluators inspect against usability principles independently before consolidating findings. Independence preserves distinct search paths; aggregation removes duplicates, clarifies evidence, and negotiates severity. Joint inspection from the outset anchors later reviewers to the first account.
Why it happens
Detection depends on expertise, path, and attention, so individual coverage differs. The union expands coverage but produces duplicate symptoms and different heuristic labels. Majority voting can erase a rare severe issue found by one reviewer; consolidation must return to interface state and user consequence.
Studying it
Fix version, scope, and heuristic set and record each evaluator's independent issue, location, and severity. Preserve provenance and disagreement while separating duplicate symptoms from shared causes. Compare with user testing for hits, false positives, and unique issues rather than ranking reviewers by count.
Where it stops holding
Additional evaluators show diminishing returns and shared backgrounds create correlated blind spots. Domain and accessibility issues require corresponding expertise. Consensus is not truth, and minority findings should not be deleted automatically.
Applying it
- Freeze individual lists before consolidation.
- Attach state, principle, consequence, and reproduction path to each issue.
- Preserve provenance and disagreement when merging symptoms.
- Validate priorities through user testing and post-fix review.