Problems fitting no entry should be kept; they show the current set is incomplete

What it is

Real users and evaluators find flaws no checklist has named: insufficient explanation of an algorithm's output, a broken handoff across devices, confused permission semantics inside a team, unexpressed model uncertainty. An unclassified finding should not be discarded for lacking a label; it is a signal of a blind spot in the heuristic set. This card completes the loop with the previous three: those handle what to do when one problem gets multiple labels, when labels get double-counted, and when there are too many labels; this one handles the reverse situation — a problem that fits under no label at all, where the fault usually lies with the checklist, not the problem.

Why it happens

A generic heuristic checklist is, at bottom, an inductive summary of problem patterns that have already been observed and validated repeatedly in the past, and it inherently lags behind changes in technology, business, and platform — a checklist written in the era of graphical interfaces will not naturally contain a problem type like "why did the algorithm produce this result, and how confident is it," which only emerges once automated recommendation or generative features are introduced. Whether a specific problem is valid depends on whether it has real evidence and consequences, not on whether the existing taxonomy happens to have reserved a slot for it — if a finding gets deleted or ignored just because no suitable label exists, the team is effectively letting the checklist's historical limitations decide what counts as a problem in the first place, which reverses the actual causality. Keeping unclassified findings also has a longer-term payoff: if the same kind of unclassifiable phenomenon keeps recurring across multiple products and tasks, that recurrence is itself a signal that the current heuristic set no longer covers the interaction forms in use, and it is worth distilling a new check item out of this pile of repeated unclassified records.

Where it stops holding

Not every one-off unclassified problem should be promoted into a formal rule. Before deciding to write it up as a new entry, confirm it has sufficient evidence, frequency, and impact, and check whether it generalizes beyond the one product it was found in — if it appeared only once in a single review, it may well be an individual preference or an isolated defect, and institutionalizing it as a general rule would unnecessarily lengthen the checklist, recreating the "too many entries" problem discussed in the previous card. A separate case needs distinguishing too: some problems are already covered by an existing entry, but the evaluator simply failed to recognize the match — the root cause here is insufficient training or unclear entry wording, and the fix is to improve the entry's description and training material, not to mistakenly conclude the checklist has a gap and add a redundant new entry.

Applying it

• Always keep an "other/unclassified" category in the problem-record template, and require records under it to fill in the same factual fields as any other problem — triggering situation, object involved, actual consequence, supporting evidence — just without a matching label yet.
• Periodically review all unclassified records to check whether a recurring triggering situation or consequence pattern shows up among them, and judge whether it has accumulated enough weight to justify a new checklist entry.
• Before formally adding any new entry, pilot it as a trial check across several different tasks and product contexts to verify its hit rate is stable and its false-positive rate is manageable — do not write a new entry into the formal list the first time a single unclassified record appears.
• How to check: after a new entry goes live, track whether its hit rate shows a real improvement over the previous checklist version; also sample a batch of historical records that were "actually covered by an existing entry but not recognized by the evaluator" — if this kind of misjudgment is common, the problem lies in training and wording, and the entry description should be fixed first rather than papering over the real cause with more new entries.

Related

• Same group: B3.18.1 The same problem often fits several heuristics; classification disagreement does not invalidate the problem · B3.18.2 Heuristic entries are not mutually exclusive; counting problems per entry double-counts · B3.18.3 Too many entries reduce the proportion actually used by evaluators
• Nearby: Q4 Research Methods and Evaluation · Q2 Usability Evaluation
• Search terms: unclassified finding · heuristic gap · emerging pattern · checklist drift