L2.01.6magic-phrase superstitiondesignresearch

Synonymous phrasings yield different results, so users invent phrases to memorize

Aliases: prompt cargo cult · formulaic prompting · memorized incantation

What it is

Last week “please list the risks as a table” returned a clean grid. Today “what are the risks, put them in a table” returned a paragraph. People do not reach first for sampling and paraphrase sensitivity. They write down: you have to say “please list as a table.” Magic-phrase superstition is the folk theory that grows after synonymous sentences fork — the system does not honor intent, it honors a spell to be memorized, forwarded, and saved.

Unstable mapping is the premise. This entry is how a lucky successful wording gets promoted into procedure. Community lore of “always add this sentence” is the social form of the same thing.

Why it happens

Operant conditioning reinforces the surface form that was rewarded. One success that happened to include “please list as a table” treats the whole sentence as the discriminative stimulus, not the slot “I need a table.” To keep the result, people start copying verbatim, honorifics, punctuation, and a subordinate clause that was surplus at the time. The prompt becomes an artifact.

Social transmission speeds the fossilization. Successful sentences leave the building as screenshots more readily than failures; recipients cannot see that the sentence already died on another temperature or model version. Phrase books lengthen into ritual, while what actually transfers is structural slots. Open input never made those slots into visible objects, so ritual occupies the vacancy.

Studying it

First let people live through “same meaning, different result” (a paraphrase pair plus a sampling control). Then watch how they write personal rules for “how I will ask next time.” Code the rules as slot rules (“specify the format”) or literal rules (“these five characters must appear”). Independent variables: side-by-side display of paraphrase and result, whether the successful sentence is parsed into editable slots, presence of community slogans as distractors. Dependent variables: verbatim copy rate, whether the rule is still obeyed after a model version change, whether retellings to another person hand over the original sentence or the structure.

Watch practice effects: three paraphrases in a row and people induce slots themselves. Sparse use — once every other day — is the ecology where superstition sets.

Where it stops holding

Internal system prompts and role preambles are engineering objects meant to be reused verbatim; treating them as phrases is not superstition. Code completion and slash commands already trigger on reserved words; a fixed saying is grammar. In creative writing, keeping “this prefix feels right” is close to a style tool and need not be debunked. This entry is about task users treating a lucky sentence as a capability switch. The cost detonates when a model or product upgrade kills the phrase before it kills the capability.

Applying it

  • After a success, split that prompt into reusable slots rather than offering “copy the sentence you just used.” Prefill next time with format=table, not those nine characters.
  • When a user pastes an old sentence back, say that equivalent wording should yield a result of the same kind, and offer a “run it from the structure” contrast that breaks the spell.
  • Official examples should put two or three wordings of the same intent side by side, with result structure held in the same family, blocking the induction “only this sentence works.”
  • Check: ask people to change two near-synonyms in their own successful sentence and resubmit. If they refuse, or blame the drop entirely on “those two words cannot move,” the phrase has replaced the slot. A week later ask “how do you get a table out of it” and listen for the original sentence versus “specify a table.”

Related

  • Same group: L2.01.1 Open input does not cue the range of capability · L2.01.2 Not knowing how to say it is the main barrier · L2.01.3 Differences in wording produce differences in results · L2.01.4 An empty box conveys no boundary; the first sentence is a guess · L2.01.5 Open input steers failure attribution toward “I said it badly” · L2.01.7 Openness makes functions non-enumerable; the product can no longer show a complete capability list · L2.01.8 Error messages for open input stay vague because the system does not know what the user meant to do
  • Nearby: L2.07 Prompt History and Reuse · L2.03 Examples and Template Guidance · L1.01 Mismatch Between Stochastic Output and Deterministic UI
  • Search terms: magic-phrase superstition · prompt cargo cult · formulaic prompting

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/L2.01.6