Closed questions constrain the response space in advance
Aliases: closed-ended question · response-option effect · response space
What it is
Closed-question effects arise when predefined categories constrain what can be expressed. Closed formats are useful for defined constructs and distributional comparison, but they can omit unforeseen motives, mixed states, and conditional answers during exploration. Closed is not synonymous with leading; adequacy depends on whether options are exhaustive, balanced, and fit the inferential purpose.
Why it happens
Options are both response tools and cues about what counts as a reasonable answer. People may choose the nearest category rather than describe their state; missing midpoint or not-applicable choices can force a false position. Asking closed items first primes their concepts, while an initial open answer can change the reference used for later ratings.
Studying it
Cognitive interviews and pretests reveal how respondents interpret the stem, retrieve experience, and map it to options. Compare open coding with category coverage and inspect other, multiple-choice, skip, and explanation patterns. Randomization or counterbalancing can estimate open–closed order effects; option order can be randomized or fixed by a meaningful scale.
Where it stops holding
Open questions remain shaped by wording, interviewer behavior, and expressive ability, and they increase nonresponse and coding cost. Validated scales should not be casually rewritten for openness. Factual confirmation, eligibility, and genuinely binary states fit closed formats. The error is forcing an unknown domain into unvalidated categories too early.
Applying it
- Generate categories from concrete-event accounts before constructing exploratory closed items.
- Offer other, do-not-know, not-applicable, and multiple selection only where conceptually needed.
- Conduct cognitive interviews and test whether segments attach different meanings to the same option.
- When both formats are needed, choose order from the primary inference and test order effects through randomization or counterbalancing.