Semantic correctness cannot be judged automatically
Aliases: meaningful name · human judgment · cannot auto-judge semantics
What it is
The alt attribute is present; the tool passes. It says “image”; after listening, nobody knows what is in the figure. A heading mark is present; the tool passes. The mark wraps a tagline, and the reader’s heading list is still a useless index. Semantic correctness asks whether the markup lines up with the task a person has to finish. That predicate has no computable truth value. Automation cannot judge it.
Why it happens
Semantics is a relation between markup and human meaning. A checker can get the markup: whether a role exists, whether a name exists, whether the name is empty. It cannot get the meaning: whether that name lets someone recognize “submit order” in a list, whether this heading is the title of a section, whether these nodes are really tabs. “Correct” in the predicate depends on task and language, not on the DOM.
Heuristics catch some rotten templates (“click here,” “image,” “untitled”), miss sentences that look decent and still miss the task, and false-flag legitimate short names. A classifier guesses whether a name looks like a good name, not whether this name holds for this checkout. The automatic layer stops at “are the slots filled”; “is what fills them enough for a person” remains a human judgment.
Studying it
Build two pages with equally complete structural slots: one whose names and headings line up with the task, one that uses legal but empty boilerplate. The engine should pass both, or mark both for review; reader users should finish the task only on the aligned page.
Independent variables: whether slots are filled, whether names / headings align with the task. Dependent variables: engine result, whether users can name the target in a heading list, task success.
Split “alt is non-empty” and “after listening, the fact in the figure can be reported” into two dependent variables. Do not collapse them into one “alt score.”
Where it stops holding
Empty names and illegal roles are structural errors and can be decided automatically; they are not semantic problems. A language-specific denylist catches some boilerplate and dies on the next domain or language. A generative model can score “does this look like good alt” and still not hold a truth value for this figure and this user task. When a law requires “a text alternative,” a machine can prove an alternative exists, not that it is correct — correctness still has to be read by a person.
Applying it
- After an automated pass, sample names, headings, and alt text on the primary task. Listen to those strings only, and ask whether they suffice to operate.
- Boilerplate or off-task words in the heading list, button list, or link list are semantic failures, even when the tool is green.
- How to check: listen to the figure’s name and the primary button’s utterance. If the fact in the figure cannot be stated, or the button cannot be found in the button list, the semantics are incorrect — whether or not the engine complained.