Card sorting and tree testing need method-specific sample guidance
Aliases: card-sorting sample size · tree-testing sample size · method-specific planning
What it is
Method-specific sample-size planning starts from a method's data structure and inference rather than borrowing another method's heuristic. Card sorting targets clusters or item similarity; tree testing targets path success, first click, and loss points. Even though both study information architecture, their sample requirements differ.
Why it happens
Open card sorting needs enough diverse sorting schemes for structure to stabilize, and results depend on card count and labels. Tree testing estimates task-level categorical outcomes whose precision depends on success rate, tasks, repeated measures, and subgroup comparisons. Published heuristics compress assumptions from common designs and cease to apply when the objective changes.
Studying it
Specify the unit and decision: discover categories, stabilize clusters, estimate success, or compare trees. Use resampling or split halves to examine card-sort stability. Size tree tests from target interval width or minimum difference while accounting for multiple tasks per participant. Pilot data inform variance and invalid-response rates.
Where it stops holding
Numbers in tutorials assume particular card counts, tasks, populations, and tolerable error. More participants cannot repair ambiguous cards, a malformed tree, or unrealistic tasks. When comparing regions, experience levels, or assistive technologies, within-segment precision—not the total—is the constraint.
Applying it
- Define the primary metric, analysis unit, and smallest actionable difference for each method.
- Pilot invalid data, within-person correlation, and baseline rates, then update the plan.
- Report structural stability for card sorting and intervals plus task-level performance for tree testing.
- Record the assumptions behind every heuristic and reassess after design changes.