Measure readability through user comprehension rather than text length
Aliases: privacy-notice comprehension testing · empirical readability · comprehension-based readability
What it is
Empirical privacy-notice comprehension defines readability by whether target users form a correct, transferable model of data behavior, not by document length, mean sentence length, or a grade-level score. Short text can be unpredictable because it omits information; longer text can support accurate lookup through structure and examples.
Why it happens
Text metrics proxy surface vocabulary and syntax but cannot see undefined concepts, architecture, prior knowledge, consequential omissions, or mistaken product assumptions. Comprehension is a relation among reader, text, and task: one explanation may support recipient lookup yet fail on a deletion exception. Direct task tests reveal guessing and confident misconceptions hidden behind correct averages.
Studying it
Build a proposition table spanning actor, data, function, recipient, retention, rights, and exceptions. Test free recall, concrete scenario judgment, and transfer to a new case, recording confidence to separate unknowns, tentative errors, and confident errors. Include variation in language, literacy, device, and assistive technology; testing only internal legal staff greatly overestimates general comprehension.
Where it stops holding
Comprehension research is not an exam imposed on every user and should not gate service. Small samples discover faults but do not precisely estimate population pass rates. Inherently complex content may need interaction support after wording improves; failure does not automatically prescribe fewer words. Understandability also does not establish that processing is justified or lawful.
Applying it
- Derive required propositions and high-risk misconceptions from actual data flows before setting a word limit.
- Give each proposition a recall item, scenario judgment, and transfer item instead of asking only whether prose feels clear.
- Recruit by target population and context, reporting error type, omission, and confidence calibration rather than hiding vulnerable groups in a mean.
- Map every failure to wording, structure, example, or missing proposition and retest with new participants; do not substitute an automated readability score.
Related
- Same group: O2.08.1 Legalese barriers · O2.08.3 Icons and examples build mental models · O2.08.4 Simplification can omit critical limits
- Adjacent: C1.07 Readability · N1.04 Difference between self-report and actual performance
- Search terms:
privacy notice comprehension·scenario-based comprehension test·readability validity