Confidence is the model’s self-report; the user has no independent way to verify it

What it is

The number on a thermometer can be checked with a second thermometer. “80% sure” beside a generate box has no second instrument. It is the model’s self-score of this output. The user cannot open the calculation, and cannot use one sitting to falsify that 80%. Unverifiable self-report puts confidence back in its epistemic place: testimony, not measurement.

This entry does not ask whether the testimony runs high. It asks whether the person holds any instrument that could check it.

Why it happens

Verification needs an independent channel. Temperature has a second sensor; accuracy has labels after the fact; a citation has a link. An internal score’s independent channel either does not exist (logits are not exposed) or cannot be used (label a thousand items and draw a reliability diagram). So the number on the UI enjoys the look of a measurement without a measurement’s rebuttability. An irrebuttable number is taken as authority not because people are gullible, but because the procedure for opposing it does not exist.

People still try to verify: they ask again “are you sure,” or they read the tone. A second self-report correlates with the first; it is not independent evidence. Tone is another face of the same generation. The supposed check is a loop.

Studying it

Give scored output and ask “how do you know this 80% is true.” Record the verification strategies people propose, then whether those strategies can actually be run on the UI. Independent variables: whether an independent channel is offered (a source, a second method, a historical hit rate). Dependent variables: type of strategy, whether it is executed, whether trust in the score falls once they see it cannot be checked.

“I trust it” is not verification. Watch whether they can point to an observation that does not depend on the model’s self-score.

Where it stops holding

When there are external labels and the UI shows a historical hit rate, not this instance’s self-score, a verification channel exists — and that is no longer confidence, it is a statistic. Experts who can check the content against domain knowledge are verifying the proposition, not the number; the number remains unchecked. Children and people who must lean on the UI have even less of a second channel. This entry does not treat percents being read as frequency; that is the speech act of a format.

Applying it

• Label a self-score as what the model itself claims, not as a measurement. Put an independent channel beside it: a source link, a second calculation by a different method, or “we cannot check this instance’s certainty.”
• Do not offer “ask it again how sure it is” as verification. That is an echo of the same testimony.
• If there is no independent channel, consider not showing an instance score, and show something that can be checked (a source, a checker result).
• Check: point at the score and ask “how would you unmask it.” If they cannot name an action the UI affords, the number is unrebuttable testimony. Then watch whether anyone treats “it said it was sure” as having already checked the content.

Related

• Same group: L1.08.2 Percents are read as frequency promises, and most model numbers are uncalibrated · L1.08.3 Bands are less over-read than continuous numbers, at the cost of hiding within-band differences · L1.08.4 Showing uniformly high confidence on a whole batch provides no discrimination · L1.08.5 Confidence is worth showing only when the user can change the next action because of it
• Nearby: L1.04 Presenting confidence · L3.02 Source attribution · L5.01 Types of explainability
• Search terms: unverifiable self-report · confidence as testimony · no second instrument