Closed questions constrain expression but are reliable
Aliases: forced-choice prompt · closed prompt · bounded reply set
What it is
A closed / forced-choice prompt pulls the reply into a small set: “the blood-pressure medicine or the inhaler?” “third floor or the garage?” Recognition and intent become computable, and completion usually beats an open question for the same slot. The cost is clipped expression. If what the person actually needs is “both, but the inhaler first,” that act is not in the set; they pick an approximation or refuse to answer. Reliability comes from constraint, not from smarter understanding.
Why it happens
A closed question publishes a miniature grammar inside the utterance. Decoding can bias toward those words; intent labels line up almost one-to-one with the options; slot filling turns from generation into selection. Errors shift from “no idea what they said” to “heard a different legal option” — the second kind still happens, but the candidate set is closed, so checking and read-back are cheap.
The limit sits at the edge of the set. Real replies that fall between options, outside them, or that need a modifier (“inhaler first, blood-pressure next week”) have no legal place to live. People then force-fit the nearest option, say “neither,” or launch an open narrative. If the system books force-fit as success, reliability is an accounting artefact. Smaller sets are stabler and express a narrower world: a trade of recognition variance against expression bias, not free accuracy.
Studying it
Implement the same decision as forced choice and as open narrative. Log recognition / intent accuracy, the rate that cannot be mapped onto an option, restatements, and a post-task check: “what did you actually mean” versus the value the system took. Independent variables: number of options, whether they are mutually exclusive, whether “neither / both” is offered.
The contrast that matters is “heard correctly, picked the wrong world”: the system holds a legal option the user later denies was the goal. Reporting only word error in the closed condition hides that bias. Labs that forbid “neither” overestimate how acceptable closed questions are in the field.
Where it stops holding
Where medicine or law requires people to state things in their own words (a chief complaint, an incident narrative), a closed question induces answers that look true but were shaped by the options; reliability then hurts content validity, not recognition. When options are opaque (internal codes, SKUs), the question is not actually closed and people fall back to description. Two options that cover the world cost almost no expression; five-plus with modifiers needed is where loss starts to beat stability. Offering “neither” with no next hop only postpones failure by a turn.
Applying it
- If a slot’s legal values are few and mutually exclusive, use a closed question. Speak the options in words users say, not internal labels.
- If out-of-set needs keep appearing in the corpus, give them an explicit exit (“both,” “neither”) that leads to a state that can handle them. Do not treat the exit as decoration.
- Do not use closed questions to collect narratives that must be stored verbatim. Elicit those with a short open prompt, then check key fields with a closed question on the next turn.
- How to check: sample “successful” closed-question sessions and ask “which of these were you trying to do.” Mismatch with the system’s value is expression bias, not recognition success. High bias means add an exit or split the slot, not tighten the vocabulary further.