Qualitative discovery-rate heuristics have strict conditions
Aliases: problem discovery rate · qualitative sample size · diminishing returns
What it is
A qualitative discovery-rate heuristic describes diminishing new findings when similar participants repeatedly test similar tasks and the goal is to expose relatively common problems. It can guide iterative planning but is not a law that a fixed small number represents all users. Interview themes, rare hazards, cross-group differences, and theory construction pursue different information goals and do not share one discovery curve.
Why it happens
If each participant has a roughly stable independent probability of encountering a problem, common problems appear early and the remainder becomes rarer or conditional. Actual problems are dependent: task order, moderation, learning, version, and capability change exposure. The granularity used to merge findings also changes the count. A diminishing curve therefore reflects both phenomenon frequency and study design.
Studying it
Define an independent problem, repeat occurrence, and comparable task exposure before collection. Plot cumulative findings by session and resample order to assess sequence sensitivity. Separate severity from frequency so many minor issues do not hide a late high-consequence finding. Stopping can combine several sessions without new high-priority findings, coverage of critical tasks, and coverage of planned groups rather than a participant total alone.
Where it stops holding
Discovery-rate heuristics suit formative usability problem finding, not interview saturation, population prevalence, or safety assurance. Heterogeneous participants, branching tasks, and partial task assignment create unequal exposure. Rare severe failures require hazard analysis, telemetry, or expert methods. A curve from an earlier design cannot directly predict a changed version.