Tree-test tasks should not simply repeat navigation labels
Aliases: task wording leakage · keyword matching · navigation cue
What it is
Task wording leakage occurs when a task repeats the target navigation label, enabling lexical matching instead of structural understanding. Tasks should express a realistic goal, trigger, and outcome. Avoiding direct reuse concerns artificial cues, not terminology users genuinely encounter.
Why it happens
Shared word roots create strong scent, inflate direct success, and hide whether labels fit user language. Replacing them with unnatural synonyms creates artificial difficulty. Valid wording preserves task meaning without signaling the answer.
Studying it
Have someone blind to the tree draft scenarios from real tasks, then audit lexical overlap. Cognitively pretest goal understanding and compare natural and leaked versions if needed. Separate task misunderstanding from navigation failure.
Where it stops holding
Brand names, legal terms, and authentic query words may legitimately match labels. Eliminating all overlap is neither possible nor desirable. Multi-goal tasks can fail because they are double-barreled rather than cued.
Applying it
- Draft from user events and goals rather than the target node.
- Flag shared roots and ask whether they occur naturally.
- Pretest comprehension and remove artificial cues or synonyms.
- Report task-understanding failure separately and retest revised wording.