Wrong inferences also harm, and the misclassified often have nowhere to appeal

What it is

Pushing medical ads at someone who is not a patient, pushing group content at someone who is not in the group, treating a closed life stage as current — a wrong tag still changes ranking and can still be seen by others. Unappealable misclassification means the harm of a wrong inference still happens, and users often cannot find an entry that voids the tag; they may not even know the tag exists.

Harm does not require the tag to be true. Being named and being treated is enough.

Why it happens

Classifiers have error; people in the tail take a disproportionate share of it. Once the miss enters ranking, the user meets “the system insists I am X”: the pool locks, the reason repeats a false identity. If the tag is invisible, they only feel “recommendations got weird” and cannot aim an appeal. If the tag is visible but the only control is “show less like this,” what is reduced is content, not identity; the next pass estimates it again from behaviour.

With no void path, correction cost moves onto the user: they must perform the opposite (deliberately click contrary content) or leave. Performance further pollutes the log. The error maintains itself, as rigid as a correct stereotype.

Studying it

Write a known-wrong sensitive or identity tag into some users’ profiles (under review and informed consent), against correct tags and no tag. Measure pool shift, felt misrecognition, success at finding a correction entry, whether the tag returns after correction. Independent variables: whether the tag is visible, whether a void control (“this is not me”) exists as opposed to “show less.” Dependent variables: harm from misrecognition, appeal success, time to return.

Overall accuracy will average these people away. Report on misclassified units, not only the classifier’s headline score.

Where it stops holding

Low-stakes taste misses (jazz served as pop) hurt less; a void path is still useful but not the same urgency. Fields the user typed wrong are profile errors and go through profile edit, not inference appeal. Safety blocks sometimes cannot void instantly, but they need a human review path and must not become an identity tag with no expiry. This entry is “wrong still hurts, and often has no counter.” It does not restate display-as-disclosure, and it does not treat how tolerance varies by setting.

Applying it

• Every inferred tag that changes treatment gets a “this is not me” that voids it and blocks re-estimation for a period, rather than merely down-weighting the class.
• The void must be reachable: from a weird reason line, from the profile, from a “why am I seeing this” entry.
• Check: write a false identity tag onto a test account, see whether it can be voided in five minutes, and whether it grows back from behaviour within seven days. If it cannot be voided or it returns, the appeal is empty.

Related

• Same group: L6.06.1 Behavioural inference can reach information the user never disclosed · L6.06.2 Displaying an inference is itself a disclosure · L6.06.3 Inferences in sensitive categories need extra constraints · L6.06.4 Combinations of non-sensitive behaviours can yield sensitive attributes; item-wise compliance is not whole-system compliance · L6.06.5 Inferences on shared devices and shared screens disclose to third parties · L6.06.7 Tolerance for inference depends on the setting; the same inference is judged differently across products · L6.06.8 Opting out of inference and opting out of data collection are different controls
• Nearby: L6.10 Turning Personalization Off and Resetting It · L6.13 Negative Feedback Channels for Recommendations · L6.07 Presenting Recommendation Reasons
• Search terms: unappealable misclassification · inference error harm · this is not me