After assistive tech takes over, the gesture layer dies
Aliases: screen-reader gestures · semantic layer · accessibility tree
What it is
With VoiceOver, TalkBack, or Switch Control on, the system reinterprets touch as its own gestures: one-finger swipe moves focus, two-finger swipe turns the page. App recognizers often never see the original stream; the gesture layer dies. Functions must still be reachable through activatable nodes in the accessibility tree, accessibility actions, or equivalent controls. That is not another single-finger geometric substitute; it is hanging the command on the semantic layer.
Why it happens
Assistive tech inserts its own interpreter between the driver and the app. The app sees translated focus movement and default activation, not raw multipoint paths. Pinch-zoom, swipe-to-archive, and long-press drag have no counterpart in that pipe. If a command lives only in touchesMoved, the screen-reader object tree is missing it. The right exposure is: every state change a gesture can fire also has a labelled element or action. The system eating app gestures is not an in-app disambiguation bug; it is a layer change.
Studying it
Walk primary tasks under a screen reader and under switch control. Record which commands vanish, whether focus lands on an equivalent control, and whether custom actions are spoken. Diff the visible gesture list against the action list in the tree; the gap is the semantic hole. Participants should be everyday AT users; sighted people wearing a screen reader overestimate exploration strategies.
Where it stops holding
Marking a control tappable while leaving the real commit in a gesture callback makes the tree look complete and the function still dead. A full-screen game canvas with no AT exit goes mute at this layer. An external keyboard uses another focus system, not the touch gesture layer, but must still reach the same commands. Semantic reachability does not replace adjustable thresholds: someone who can hear a button may still be unable to hold a long-press.
Applying it
- For every gesture command, put a named action or visible button in the accessibility tree that a screen reader can list and activate.
- Do not assume the app still receives the raw swipe; with a reader on, expose swipe-to-archive as an “Archive” action on the focused item.
- Run purchase, archive, and reorder with VoiceOver or TalkBack; any step that still requires the original app gesture is a missing semantic path.