Breaking the loop needs active exploration; the cost is immediate and the benefit is delayed

What it is

To learn whether an unshown item is good, it has to be shown first. That showing is the turn users will mostly experience as “less accurate than before”: clicks fall, complaints rise. Only after some sessions can the model use the observation to open the list. Exploration cost now, benefit later is that time structure: breaking the exposure loop bills an immediate experiential cost, which may be recovered later in coverage and discovery.

An external signal supplies a breakpoint. What is nailed here is the temporal structure of the bill, and why products systematically underpay it.

Why it happens

Exploration–exploitation writes each display as: exploit the current best, or explore to cut uncertainty. Exploit’s reward enters clicks at once; explore’s reward is information, and information is discounted. Users discount more steeply — they live this screen, not a regret bound. Explore slots on the interface are therefore read as a downgrade, and operations close them with clicks.

Information belongs to the system; discomfort belongs to the user. If the discomfort is unexplained, not limited to a skippable seat, and not cashed out as later improvement, people will not agree to keep paying. Epsilon-greedy and Thompson sampling can lower regret in simulation and still fail to handle “why did this screen suddenly go off.” HCI has to make exploration an intelligible trade, not lay the estimator on the first screen.

Studying it

Open explore slots on users in a steady state. Record immediate clicks, felt downgrade, and coverage and discovery (new classes tapped) over later sessions. Independent variables: explore share, position (first screen vs end), whether labelled “try something new,” whether skip is allowed and not stored as negative. Dependent variables: immediate cost, delayed return, rate of turning exploration off mid-way.

A regret curve describes efficiency on the system side. Immediate user ratings must sit beside it. Reporting only long-run nDCG hides the present bill, which is exactly why exploration dies in the product.

Where it stops holding

When users walk into a Discover rail, they have prepaid the cost; the first screen need not also levy it. When safety and inventory already bound the explorable set, exploring at random is harm. Cold-start exploration is there to get a first observation; the cost structure is different (there is no accuracy to exploit yet). This entry is the time structure. It does not restate exposure bias, and it does not restate offline evaluation’s bias toward the logging policy.

Applying it

• Put exploration in a skippable, labelled seat: “try one you may not have seen.” Do not mix explore items into the same “for you” promise as the main sort.
• Do not store a skipped explore item as a negative; store information only on open-and-complete.
• Check: open exploration for two weeks. First-week clicks may fall inside a written budget, but week-two coverage or new-class clicks should rise. If you only see the fall and not the recovery, the share is too high or the return was never cashed out to the user — move the slot off the first screen or make the label explicit, then retest.

Related

• Same group: L6.09.1 The system only observes items it showed; unshown items never receive positive feedback · L6.09.2 Early accidental clicks get amplified into durable profile features · L6.09.3 Users adjust behaviour to get the recommendations they want, so behavioural data lose representativeness · L6.09.5 Offline evaluation on historical logs systematically favours the policy that produced those logs
• Nearby: L6.04 Feedback Loops · L6.08 Filter Bubbles and Diversity · L6.02 Filter Bubbles
• Search terms: exploration-exploitation · exploration cost · delayed diversity return