Personalized ranking must be labeled as customized, not as the general result

What it is

The same query can yield a different top ten for a signed-in user than for a signed-out session or another account: history, location, role, and clicks have been written into the ranking. Labeling the list as customized means saying “this roster is for you,” and offering a path to the general roster. Unlabeled, people treat a private ranking as the world’s objective order, argue from it that “searching this ought to return that,” or assume the object is equally findable for everyone. Pitkow, Teevan and others on personalized retrieval keep hitting this transparency problem: once customization is invisible, it is read as neutral.

It is not a dump of the profile, and not turning personalization off. It requires that the kind of list be distinguishable: general, or ranked for you.

Why it happens

A ranking is read, by default, as a public answer to the query. Personalization folds private variables into that answer without changing the list’s appearance, so public and private are isomorphic in look. Later behaviour follows the public reading: forwarding a link and assuming the other person sees the same row, training query habits on the top three, narrating “can’t find it” as “it doesn’t exist.” The other person’s list differs; habits have been shaped by private variables; the content sits in the general list.

A label reclassifies the roster as “a view aimed at me.” Once that class exists, people expect others to see something else, and they go looking for an uncustomized contrast. Without a contrast, the label is a badge and cannot be used to inspect what customization actually changed; personalization remains unaccountable.

Studying it

Compare silent personalization with “labeled, plus a general contrast” on judgments of publicness and on query strategy.

• Paradigms: one query, a customized list and a general list; one condition never names the difference, the other marks “ranked for you” and allows a switch. Ask “what would a colleague see for the same sentence,” and whether people switch in order to check. Logs can show use of the switch; non-use does not mean it is unneeded—it may be undiscoverable.
• Independent variables: whether customization is labeled, whether a one-click general list exists, whether the signals are summarized in one sentence (“based on projects you open”).
• Dependent variables: accuracy of publicness judgments, rate of treating a private top three as everyone’s top three, switches made in order to check.
• Methodological note: if two people never compare screens, errors about publicness are hard to experience. Use at least a pass-along task: describe the results to the next participant. Do not take satisfaction as success: silent customization often feels “smoother” and misleads more.

Where it stops holding

Ranking with no personal signal (signed out, pure keyword, no history) must not be labeled “customized for you”—that invents a privacy that is not there. Collaborative retrieval, shared screens, and training sessions pay the highest cost for customization; default to the general list or auto-downgrade. A recommendation feed (“guesses for you”) is already read as private, and a second label adds little; a query issued from a search box is still read as a public question and must be labeled. On child or supervised accounts, customization may not be something the person can turn off—but the list still has to be marked as not a neutral corpus ranking.

Applying it

• When history, role, location, or clicks change the ranking, put stable copy on the result page: “ranked for you,” or “weighted toward your workspace.”
• Offer “view uncustomized results,” or an equivalent switch. Query string and filters stay put across the switch.
• Summarize signals at a layer people can recognize (“your recent projects,” “your region”). Do not list feature weights.
• Verify with two accounts searching the same sentence, different top threes. Each person should be able to say the list is customized, and to switch to the signed-out order. If the two lists look identical and cannot be switched, customization has been passed off as the general result.

Related

• Within the group: G3.17.1 Results should say whether the match was in the title, body, tags, or another field · G3.17.3 Implicit filters need their source disclosed · G3.17.4 Raw relevance scores are not fit to show end users · G3.17.5 Explanations exist to help reformulate the query, not to prove the ranker is right
• Adjacent: G3.05 Result ranking · G3.11 Search history · O1.03 Purpose limitation
• Search terms: personalized search · ranking transparency · customized results