A human performs capabilities the system has not yet implemented
Aliases: WOZ · human-enacted system capability
What it is
Wizard of Oz has a hidden person perform capabilities the system has not yet built: understanding speech, recommending a next item, recognizing an image, generating a reply. Participants face an interface that looks like a system; decisions are made by an operator. Kelley chose the name to mark that the magic comes from someone behind the curtain, not from an already existing recognition or inference engine. It is not a usability facilitator “clicking for you”: the enactment is the experimental condition, must follow a script, and is usually invisible to the participant.
Why it happens
Many interaction hypotheses depend on capabilities that are not yet stable: whether speech survives noise, whether a recommendation earns trust, whether generated text can be edited. Waiting until the capability runs binds interaction questions to model questions. Wizard of Oz replaces the capability layer with a flexible human, so interface, turn-taking, and error handling can be observed first. People can improvise branches an algorithm does not have yet—an advantage and a contaminant. Operators understand context and complete better than a model. Enactment must therefore be constrained to the set of behaviors the future system will be allowed; otherwise the study tests a clever assistant, not the target system.
Studying it
The classic setup is a one-way mirror or remote console: participants see only the front end; the operator selects outputs under recognition/decision rules. Factors can include the scope of enacted capability, allowed delay, and injected error rate; outcomes include task success, repair strategies, trust, and whether the human is detected. Dahlbäck, Jönsson, and Ahrenberg insisted on written operator rules, or the study cannot be replicated. Record every intervention and, in analysis, separate in-rule enactment from improvised overreach. Debrief is the ethical default.
Where it stops holding
If participants detect the operator and start talking to that person, the data no longer represent human–computer interaction. Highly specialized domain tasks may exceed what an operator can do in real time, injecting errors or unacceptable pauses. Error types the model must produce itself—hallucination, miscalibrated confidence—are hard for a person to simulate stably. Long, unattended use cannot be staffed. In legal or safety settings, hiding a human decision-maker may be unacceptable; switch to an explicit human–machine collaborative prototype.
Applying it
- Write an operator manual first: allowed outputs, forbidden completions, how failure should look when the operator does not know.
- Make enacted capability a visible system boundary (for example, “only these three request types”) so the operator cannot become infinitely clever from common sense.
- Record two channels: the participant interface and the operator’s actions; mark out-of-scope interventions in review.
- After the session, disclose that a person was enacting, and ask whether that changes how they understood the system.