Screen readers don't see pixels — they query the accessibility tree apps expose

What it is

Assistive technology on Windows does not recognize buttons from screenshots. It queries a tree the app exposes through a system accessibility API—today, UI Automation (UIA). Each node needs a name, a control type, a value, and state, and it must implement matching control patterns: Invoke if it can be activated, Toggle if it can be switched, RangeValue if it can be set. Magnifier, Narrator, third-party readers, and UI automation tests walk the same tree. An owner-drawn control that only looks like a button, without a node in that tree, leaves Narrator with nothing to say.

This is whether the system tree is complete, not which legal tier you must meet, and not how semantic markup is written on the web. System window controls usually grow the tree for you. DirectX, owner-drawn HWNDs, and cross-platform immediate-mode engines make the app itself the provider.

Why it happens

Assistive tech and the window are separated by the system. The app is the provider, mapping visual objects to UIA nodes; Narrator is the client, reading names, exercising patterns, and setting values on command. If the mapping is missing, the client cannot invent it from pixels—no name means unnamed, no Toggle pattern means a switch is read as a picture. Patterns are stricter than role: role says what it is, patterns say what can be done to it. A slider that looks draggable but does not implement RangeValue cannot be adjusted by Narrator, and a test cannot set its value.

Live changes must be pushed as property-changed and structure-changed events. Painting a new value without an event leaves Narrator on the old one. Keyboard focus and UIA focus should name the same object, or the highlight the keyboard user sees and the node the reader speaks will drift. That is the value of system controls: they already wire name, patterns, and notifications into the tree. What the app is spared is drawing; what it keeps is the query channel.

Where it stops holding

Purely decorative icons, separators, and brand illustration should stay off the tree or they become reader noise; nodes belong to objects that can be operated or that must be known. Remote desktop and some fullscreen games replace the tree with one large image for a while; UIA cannot help then, and the app needs its own speech hook or must leave exclusive mode. A web view inside a window has the engine’s own accessibility tree; the join with the host UIA tree is where focus is lost or announcements duplicate. Tests that only compare pixels are not testing whether this API is connected. The Name should be a word that means something to a person, not an internal control id—the API will speak whatever string you put there.

Applying it

• Prefer system controls. Owner-drawn controls must act as UIA providers: fill Name, control type, and state, and implement the patterns the control actually supports.
• Fire notifications for focus, selection, value, and tree inserts/deletes. Wherever keyboard focus lands, UIA focus lands.
• Do not use internal variable names as Name, and do not hang decorative nodes on the tree. Icon-only buttons get a name that matches the visible label they would have had.
• Verify with Narrator, eyes off, through the main task: every operable object is spoken as the right type and name, switches can be toggled, sliders can be set. Then inspect the tree. Any “button on screen, no node in the tree” or “node without the matching pattern” is a gap in the API. Walk the same path from the keyboard and confirm the focus highlight and the spoken node remain one object.

Related

• Same group: R4.03.1 Windows, menus, and shortcuts form one command system · R4.03.2 Mouse, keyboard, and touch must be accommodated together
• Adjacent: R3.09 Semantic structure and accessibility implementation · J1.04 Legal requirements
• Search terms: UI Automation · control pattern · automation tree · Narrator