Primary mid-air states need at least one channel that is not vision

What it is

Mid-air gestures often happen while the eyes are already busy: on television content, on a surgical field, on the road. Once contact feedback is gone, painting primary states (in range, fired, dropped) only in a screen corner asks people to tear gaze off the task. At least one non-visual channel—sound, a wrist or fingertip buzz, bone conduction—must be able to carry those states on its own. This is not re-acting every one of three states in audio; that is encoding. It is: after contact is removed, the substitute must not all be stacked on the pair of eyes already in use.

Why it happens

In these settings vision is the task channel, not an idle feedback bus. Moving gaze from content to a corner indicator costs a saccade that surgery and driving cannot pay. Hearing and touch have the temporal resolution to pin a fire edge, and they do not take gaze. Their spatial resolution is poor, so they are a bad way to say “which target is locked,” and a good way to say the low-bandwidth states “I see you / it clicked / it dropped.” A visual-only substitute looks closed in a demo because people are staring at the indicator, and opens the moment the real gaze target appears. Blind users never had the visual substitute.

Studying it

Occupy gaze with a primary task (track a moving target, watch a clip, stare at a work zone) and issue mid-air commands in the periphery. Conditions: visual state only, audio only, haptics only, combinations. Measures: primary-task errors, command retries and misses, whether gaze is pulled to the indicator (eye tracker). Noise and gloves as stress. Do not test the necessity of a non-visual channel with a huge status lamp in the middle of a blank screen—that empties the visual channel.

Where it stops holding

When the eyes are already on the interface and the state is painted at the point of gaze (a headset cursor itself), vision can carry primary states and non-visual becomes enhancement. Silent places (library, ward) constrain sound; switch to haptics. Users insensitive to vibration or averse to sound need the channel to be switchable; one non-visual must not be the only one. Non-visual channels have their own Midas: status chimes that fire too often become noise and get muted, and you are back to vision-only. In driving, hearing already hosts alarms; mid-air status sounds must yield.

Applying it

• Name at least one not-by-looking signal for “seen / fired / dropped,” and accept it while eyes are occupied by the primary task.
• Allow muting sound or muting haptics, but not both down to zero, unless visual state sits at the point of gaze.
• Judge whether the non-visual channel actually closed the loop by primary-task error and gaze-stealing counts, not by “did you hear it.”

Related

• Same group: C4.11.1 Mid-air gestures have no natural contact feedback · C4.11.2 In-range, locked, and fired must be separately legible · C4.11.3 Continuous action needs continuous feedback, not only an end bell
• Adjacent: C4.26 Distinguishing three-state feedback · C7.06 Hands-free and eyes-free use
• Search: eyes-free feedback · nonvisual confirmation · haptic substitute