Percents are read as frequency promises, and most model numbers are uncalibrated

What it is

“72%” on a forecast is a contract: of a hundred days like this, about seventy-two rain. Write 72% beside a generation and people collect on the same contract. The float inside the model has usually never been settled against that bill. Percents read as frequency promises is a speech act the format brings with it. It does not wait for a misunderstanding.

That confidence is not accuracy has already been split. This splits what the “%” glyph is saying.

Why it happens

A percent sign is the grapheme of frequency. Forecasts, pass rates, battery charge trained it as “proportion under repetition.” A one-off self-score typeset as 72% is harder than a footnote; “uncalibrated” in the footnote will not win. Risk-communication work on natural frequencies works partly because the percent sign is already promising frequency. Generate products borrow the promise and have no frequency to pay.

Miscalibration makes the promise hollow. People allocate checking time at 72%: about three in four should be right, I spot-check a quarter. If true hits are 50%, the quota is systematically short. The format is signing a contract the product cannot keep.

Studying it

The same latent score as %, as “about seven in ten,” as “fairly sure,” as “uncalibrated internal value.” Ask “if there were a hundred items like this.” Independent variables: the sign, whether “this is not a forecast” is written next to it. Dependent variables: whether a frequency answer is given, the spot-check quota that follows, feeling cheated after true hits are revealed.

If the natural-frequency arm still collects as frequency, the problem is semantic not glyphic; if only the % arm does, the problem is the grapheme.

Where it stops holding

A probability published under forecast rules, with public calibration, can bear a percent sign — it is actually signing a frequency contract. Battery and progress % promise capacity or completion, not hits; borrowing them onto confidence adds another layer of mess. Band words dodge the percent sign, though “likely” will still be translated by some people back into “about eighty”; that is the next card’s cost. This entry is only % as a promissory format.

Applying it

• Do not use % for an instance self-score. Use words, or “this is not seven in ten correct.”
• If you truly have frequency evidence, write the denominator and the window: “of the last thousand invoice fields, 720 were right.” That percent can be paid.
• A footnote “for reference only” will not cover a %. In tests the footnote almost always loses to the glyph.
• Check: give only “72%” and take dictation of what it means. If “seventy-two in a hundred” appears and you have no such frequency, the format is writing cheques. Then look at how much checking time they reserved for this class — reserved at 72% is not enough for a 50% truth.

Related

• Same group: L1.08.1 Confidence is the model’s self-report; the user has no independent way to verify it · L1.08.3 Bands are less over-read than continuous numbers, at the cost of hiding within-band differences · L1.08.4 Showing uniformly high confidence on a whole batch provides no discrimination · L1.08.5 Confidence is worth showing only when the user can change the next action because of it
• Nearby: L1.04 Presenting confidence · L1.03 Visualizing uncertainty · L5.03 Trust calibration
• Search terms: percent as frequency promise · natural frequency · uncalibrated percent