Thumbs up and down collect satisfaction, not correctness

What it is

A factually wrong reply is fun to read and gets a thumb up. A correct but wordy reply gets a thumb down. Thumbs collect “how this pass felt,” not “whether this sentence is true.” Thumbs ≠ correctness requires that when thumbs are used as a quality signal they be treated as preference and affect, not as a fact check.

Negative feedback with no location cannot localise — that is grain. Here even the polarity does not mean true or false.

Why it happens

A thumb is a near-zero-cost affect control; it invokes “liked / disliked.” Correctness is a judgment against an external record, costly, and often inverse to liking: corrective, hedged, or refusal outputs are more correct and less pleasant. Training or product metrics that eat thumbs directly will reinforce pleasantness and punish correctness.

People also do not decompose inside a thumb. One down may be a fact error, a tone error, too long, the wrong genre. The system sees only −1. Treating −1 as an error label crushes a bundle of causes into a fake fact check.

Studying it

The same outputs: people give thumbs; others (or a gold standard) label true/false. Report the correlation. Independent variables: funniness, wordiness, whether a question that should not have been answered was refused. Dependent variables: thumb–correctness correlation, error rate among upvoted, correctness among downvoted.

A correlation near zero or negative is the signature. Do not use “which side users like” as model selection without splitting out correctness.

Where it stops holding

Jokes, copy, melody have no external true/false; thumbs are the right target. On a closed right/wrong item, replacing thumbs with “is this right” plus a forced comparison collects correctness — the control is already not a thumb. Expert review queues can take both thumbs and correctness labels, but training must keep the channels apart. This entry does not treat who bothers to click (self-selection).

Applying it

• Do not feed thumbs straight into a factual loss. Factual tasks need a separate true/false channel or a check, not up/down to update “correctness.”
• Write the meaning of thumbs in human language: “did this help you,” not “is this right.”
• Split reports: satisfaction, accuracy. Do not sum them into one “quality score.”
• Check: sample upvoted and downvoted items for facts. If errors among upvotes are not low and correctness among downvotes is not low, thumbs are not correctness. Switch the loss to eat only checked true/false and see whether error rate falls.

Related

• Same group: L3.13.2 Negative feedback that does not point at a location cannot localise the problem · L3.13.3 People who submit feedback are a self-selected few; extreme experiences are over-represented · L3.13.4 Edits are implicit feedback with more information than an explicit rating · L3.13.5 If feedback produces no visible change, submission decays toward zero
• Nearby: L3.12 Editing and Taking Over Generated Content · L6.13 Negative Feedback Channels for Recommendations
• Search terms: thumbs up · satisfaction vs accuracy · preference not fact