Random injection is not structured diversity; it only adds noise

What it is

Draw a few items at random from the catalogue onto home, and mean similarity falls, but what users often see is “this is wrong.” Random versus structured diversity means unstructured injection only raises noise. Diversity is a planned spread on a chosen dimension — different authors, different stances, uncovered classes — using methods such as maximal marginal relevance, determinantal point processes, or quota slots, not a shuffle.

Once diversity is designed into the objective, you still have to decide which design. A shuffle is not one.

Why it happens

Uniform random from the whole catalogue draws a mix of head titles and wholly unrelated tail. Head titles do not add coverage; the tail is read as a fault. Structured methods name a dimension first, then maximise spacing on it or fill a quota. What they inject is “a legitimate point on this dimension you have not seen,” not an arbitrary point.

Noise pollutes the evaluation of diversity: users book the fault to “diversity,” and the product retreats to pure relevance. Random also has a statistical problem: expected coverage lifts slowly and variance is high, so a short online test will call it harmful. Structured quotas buy stabler coverage on the same slot budget.

Epsilon-random in exploration–exploitation exists to give unshown items a chance to be clicked; its purpose is estimation, not a meaningful menu. Treating the epsilon slot as the diversity experience is putting an estimator on the interface.

Studying it

Three injections, same slot budget: uniform random, popularity-biased random, structured (quotas or MMR-style). Measure: coverage of class / author / stance, rate of reading injected items as error, short-term clicks, whether people can say “the system is showing me different X.” Independent variables: whether a dimension is named, whether the slot is labelled. Dependent variables: coverage efficiency (new classes per slot spent), false fault marks.

Do not take intra-list embedding distance as success on its own. Random can stretch embeddings too. Ask whether the stretch is on the dimension written down in advance, and whether people recognise it.

Where it stops holding

Cold-start probing, ad fill, and replacing a title pulled for rights reasons sometimes leave random as the only remaining method; label it “a look around,” not “discover.” When the catalogue is tiny, structured quotas cannot fill and collapse toward random. This entry distinguishes two injections. It does not discuss how many clicks to spend, and it does not choose the dimension.

Applying it

• Write the dimension and the fill rule on diversity slots. Ban “draw N at random from the catalogue” as the implementation of a discover rail.
• Keep random slots used for probing separate from diversity slots shown as experience: probes need not explain themselves; a diversity slot should let people say “different in what.”
• Check: run a random rail and a quota rail on the same number of slots for a week. The quota rail should cover more on the named dimension and be judged error less often. If the two feel the same, the quota rule is not actually working.

Related

• Same group: L6.08.1 Narrowing is produced by the objective itself; no one has to intend it · L6.08.2 Diversity costs short-term clicks, so it has to be an independent objective · L6.08.3 Users cannot evaluate content they never saw; absence produces no visible feedback · L6.08.5 Diversity must sit on dimensions the user cares about; cross-category mix does not fix a single viewpoint
• Nearby: L6.02 Filter Bubbles · L6.09 Feedback Loops and Preference Entrenchment · L3.01 Generating Multiple Options
• Search terms: random versus structured diversity · MMR · determinantal point process