Heuristic entries are not mutually exclusive; counting problems per entry double-counts

What it is

Heuristic entries are inspection lenses, not a mutually exclusive taxonomy. The same flaw can land under feedback, error prevention, user control, or help depending on which evaluator wrote it up; summing "how many problems were found under each heuristic" produces an inflated total and misjudges the quality of a given area. This card builds on the previous one, "classification disagreement does not undermine the problem itself": if that card settled whether tagging the same problem two different ways is an error, this one handles the next question that immediately follows — once multiple tags are allowed, how do you aggregate them without counting one flaw as several.

Why it happens

Problem count should be tallied by the combination of a unique triggering situation plus its actual consequence, not by how many times a tag appears — these are two entirely different units of counting, and mixing them is the root of the error. If three evaluators independently label the same upload failure as a "state visibility problem," an "insufficient feedback problem," and an "error recovery problem," it is still three observational angles on one underlying flaw, not three separate flaws. Summing by tag count produces the false conclusion "three problems were found," which then gets scheduled as three units of work, when fixing the one trigger point resolves everything recorded under all three tags. This inflation is not merely a cosmetic numbers issue — it systematically distorts cross-module comparisons of "which area has worse quality": a module with genuinely fewer problems, each of which happens to attract multiple tags, will look like it has more problems and worse quality in a tag-counted report, and a team's resources can end up misdirected toward an area that did not actually need priority.

Where it stops holding

Not every similar-looking problem should be merged. Two problems with different triggering objects, paths, or permissions, even if they present the same surface symptom ("clicked the button, nothing happened"), can stem from two completely different underlying causes; forcing a merge loses each one's real frequency and scope of impact, diluting a genuinely high-frequency issue into a merged record that looks less common than it really is. Merging requires the evidence chain to be strong enough to confirm a shared underlying cause — surface similarity alone is not sufficient grounds. When evidence is insufficient, the safer move is to keep the records linked but separate, rather than rushing to delete one of them. Counting problems by interface module also needs care: modules differ in size and complexity, so raw problem counts are not naturally comparable across them — a larger module having more problems may simply reflect that it carries more functionality, not that it is worse quality.

Applying it

• Assign each problem a unique ID, recording the triggering situation, involved object, actual consequence, supporting evidence, and the set of applicable tags — multiple tags are allowed, but there is only one ID.
• When aggregating a report, first deduplicate by situation-outcome combination, merging multiple tagged records under the same underlying cause into one primary record; choose a primary tag to route the fix, and keep the rest as linked information.
• Present the deduplicated total problem count, the tag distribution, and a tag co-occurrence matrix in the report — never sum the per-tag counts as the total number of problems.
• How to check: periodically sample records with high tag co-occurrence and manually confirm whether they really point to the same underlying cause; if so, the aggregation process is working. If a merged record turns out to come from different trigger paths, split it back apart rather than letting a bad merge sit and skew subsequent statistics.

Related

• Same group: B3.18.1 The same problem often fits several heuristics; classification disagreement does not invalidate the problem · B3.18.3 Too many entries reduce the proportion actually used by evaluators · B3.18.4 Problems fitting no entry should be kept; they show the current set is incomplete
• Nearby: Q2 Usability Evaluation · Q4 Research Methods and Evaluation
• Search terms: double counting · usability findings · problem aggregation · deduplication