Expert review is inexpensive but has expert blind spots
Aliases: expert blind spot · curse of knowledge · familiarity bias
What it is
Expert review avoids recruiting and moderating many users and is often quick to start, but creates expert blind spots: practiced reviewers struggle to reconstruct how people without terminology, permissions, or work experience interpret an interface. Lower collection cost is not broader evidence.
Why it happens
Procedural knowledge becomes automatic, hiding intermediate inferences. Experts tolerate dense information and complex paths. Internal experts normalize historical compromises; external experts miss local constraints. Homogeneous reviewers reproduce correlated blind spots.
Studying it
Compare predictions with novice, peripheral-role, and field performance and classify misses and false positives. Require reviewers to state assumed knowledge and uncertainty, and inspect incremental findings from different expertise mixes. Include preparation, consolidation, and validation in cost.
Where it stops holding
Blind spots do not invalidate expertise. Rare hazards, standards, and complex rules often require specialists, while users may not anticipate system consequences. Expert and user evidence should support different claims.
Applying it
- State prior knowledge and user role behind every recommendation.
- Add novice, assistive-technology, and frontline perspectives.
- Fix clear rule violations and validate disputed experience claims with users.
- Include downstream validation and rework in cost estimates.