Voice earns its keep when hands and eyes are already taken

What it is

A dialogue system charges for every turn: time, recognition error, social exposure. That bill is worth paying only when hands and eyes are already locked by a primary task — wheel in both hands, spoon in a pan, bags at security. The GUI's finger and glance cannot be spared, so voice becomes the main path rather than a flourish. Sitting at a desk, screen lit, “ordering lunch by voice” can work as a channel; the overhead of conversation is not offset by occupancy. The claim is about whether a whole dialogue is worth opening, not about the microphone as a physical substitute for touch.

Why it happens

Graphical interfaces trade space for time: options are laid out together, one tap commits. Dialogue serializes the same job: speak, wait, be understood, maybe confirm. Every beat of that serial chain competes with the primary task for attention. When hands and eyes are free, the serial chain is a net loss against “glance and tap.” When they are locked on a wheel, a knife, a heading, the GUI's fixed cost — free a hand, tear gaze off the road — rises above the conversational cost, and voice starts to win. Value tracks how hard the occupancy is, not how accurate the recognizer is: perfect ASR still loses to a screen if nothing is occupying the body.

Occupancy splits. Dough on both hands with a wall screen still in peripheral vision is mostly hands-free value; night cycling with eyes glued forward but a thumb free is mostly eyes-free value. Bundling both as “voice is convenient” counts scenes that only liberated one end as successes.

Studying it

Dual-task setups lock a hand or an eye on a primary task (driving simulator, assembly, cooking walkthrough) and have people complete the same goal by dialogue versus touch. Dependent measures are primary-task degradation, eyes-off-road time, and number of dialogue turns — not dictation word error in a quiet lab.

A more ecological cut is a context diary: for each wake, what were the hands and the gaze doing. If most sessions happen on a sofa facing a screen, the product's “voice-first” story does not match occupancy. Wizard-of-Oz dialogue can be run under occupancy before a recognizer exists, to measure channel choice rather than acoustics.

Where it stops holding

People who rely on dialogue as their only usable channel when hands and eyes are free are not in this occupancy story; the value is access. Social cost in public can swamp occupancy gains — full hands on a train still will not speak an address aloud. Occupancy is intermittent: at a red light, hands and eyes briefly release and people switch back to the screen; designing the whole trip as continuous occupancy overestimates how long voice should carry a task. Dual-task labs that let people pause the primary task to look at a screen have already broken the occupancy condition.

Applying it

• List the situations where a hand or an eye is truly locked. Make dialogue the main path only for goals that occur there. Desktop and sofa capabilities are optional shortcuts; do not accept them against a main-path rubric.
• For driving, kitchen, carrying: the pass condition is hands staying on the primary task and eyes-off-road below a written cap, not “voice can also finish it.”
• Split hands-free and eyes-free in the use cases. Hands only: a flicker of glance may be allowed. Eyes only: a thumb tap may be allowed. Both occupied: no look-at-screen step.
• How to check: walk the real car or kitchen with first-person gaze and both hands on camera. If a hand leaves the primary task or gaze leaves the environment while the goal completes, occupancy value has not been delivered.

Related

• Same group: M1.01.2 Voice prefers short commands over long procedures · M1.01.3 Comparison and browsing tasks are a poor fit for voice
• Nearby: M1.02 Memory load of screenless interaction · C7.06 Hands-free and eyes-free use · M4.01 Social cost of using voice in public
• Search terms: occupied-task voice value · hands-busy · eyes-free dialogue