Users adjust behaviour to get the recommendations they want, so behavioural data lose representativeness

What it is

When someone wants one class and keeps being served another, they may tap the wanted class in a streak, deliberately skip the unwanted, or search a keyword to “teach” the system. The log looks like taste changed. It was an operation on the model. Performative preference is behaviour produced to change the next display, not as a response to the content itself, and so it can no longer be treated as an unbiased sample of latent taste.

A mis-tap is noise. Performance is strategy. Both write into the profile; they do not come from the same place.

Why it happens

Once people understand “tap and it returns,” a click becomes a tool: tap is a vote, skip is a veto, search is a prompt. Instrumental clicks are generated conditional on the model, which violates the learning assumption that behaviour is independent of the current policy. The more transparent the model (the more specific the reason), the easier the tool; the more opaque, the coarser the heuristic (“maybe if I swipe more it will change”), and the dirtier the log.

Performance also self-confirms: positives acted into being summon more of the class, the user gets the list they wanted, and they keep acting. The observed distribution locks on a strategic equilibrium, not on unmanipulated taste. Fit “true preference” on that log and you fit the equilibrium.

Studying it

Pair interviews with logs: ask “have you ever tapped or skipped on purpose to change recommendations,” then mark those stretches in the log. Experimentally, give one group an explicit control (“more like this”) and another only indirect control via clicks; compare divergence between behaviour and self-rated taste. Independent variables: whether an explicit control exists, whether reasons name the basis. Dependent variables: share of instrumental clicks, self-report–log correlation, whether the list rebounds after performance stops.

Do not count “users are fluent with the product” as quality. Fluency is performance skill. Report bias in the log as an estimator of taste.

Where it stops holding

Explicit “more / less” legalises performance as control, which is good, provided it hits a feature rather than forcing people to tap content. Professional operators and merchant-farmed accounts are another kind of performance, outside ordinary users. This entry is “behaviour changed to change recommendations, so data no longer represent taste.” It does not treat the immediate experiential cost of exploration slots.

Applying it

• Make “I want more of this class / less of this class” an explicit control, so people do not have to vote by acting.
• Down-weight or confirm short bursts of same-class clicks (“do you want to steer recommendations here?”) so strategy is not stored as taste.
• Check: find people who say they “tapped a streak to change recommendations,” and look at the weight of that stretch’s positives in the profile. If it disagrees with what they later circle as “what I actually want” and cannot be cleared, the log has already lost representativeness.

Related

• Same group: L6.09.1 The system only observes items it showed; unshown items never receive positive feedback · L6.09.2 Early accidental clicks get amplified into durable profile features · L6.09.4 Breaking the loop needs active exploration; the cost is immediate and the benefit is delayed · L6.09.5 Offline evaluation on historical logs systematically favours the policy that produced those logs
• Nearby: L6.07 Presenting Recommendation Reasons · L6.13 Negative Feedback Channels for Recommendations · L6.10 Turning Personalization Off and Resetting It
• Search terms: performative preference · gaming the recommender · strategic feedback