A change of context can void a conclusion

What it is

Even when the people look similar, a finding can stop holding once incentives, tools, organizational rules, or time pressure around the task change. Context-dependent findings bind a conclusion to the arrangement in which a relationship was observed, not to a law of human nature stripped of arrangement. A redesign, a new channel, or a move from lab to counter can make the same sentence false.

Why it happens

Many interaction results are interactions among people, interface, and setting. A lab with an observer present, no real money at risk, and tasks sliced into short sessions is not the same constraint set as a peak-hour counter, commission pressure, or a weak network. When a supporting condition disappears—defaults banned by regulation, payment switching from stored balance to real-time debit—the direction or size of an effect can reverse. Writing the finding as an unconditional “people will…” smuggles the original arrangement in as universal background.

Studying it

For each claimed mechanism, list the contextual premises it needs: time pressure, social visibility, cost of failure, network and device, organizational rewards. Use contrast or repeated observation to see whether the result survives when a premise is perturbed. Field follow-ups, log comparisons, and running the same task in a second channel test transportability; they are not a second round of “confirming what users said.” Failure to transport is a result: it marks the original claim’s range and should not be deleted as a failed study.

Where it stops holding

Findings are not equally fragile. Claims tied tightly to perceptual or motor limits often travel better under similar sensorimotor conditions; claims that lean on norms, incentives, and meaning flip more readily. Context also comes in degrees: a copy tweak is rarely a new setting; replacing payment and identity systems usually is. That the field always changes is not a reason to refuse every claim beyond the session just run.

Applying it

• Attach to each finding a short list of conditions under which the claim no longer applies automatically.
• Before a redesign or channel change, check whether those premises still hold; do not reuse old priorities by default.
• Label lab conclusions as holding under that arrangement; add targeted observation when a field decision is required.
• When new-context behavior conflicts with an old conclusion, inspect the premises before declaring that users have changed.

Related

• Same group: Q4.10.1 A conclusion’s scope is bounded by the sample · Q4.10.3 Reports must state boundary conditions · Q4.10.4 Qualitative work seeks analytic generalization, not statistical representativeness · Q4.10.5 A single-case finding is not a general law until replicated · Q4.10.6 Recheck premises before transferring across cultures or platforms · Q4.10.7 Packaging small-sample findings as universal truths is a common overreach
• Adjacent: Q1.10 Three-dimensional classification of methods · Q4.06 Scenarios and storyboards
• Search terms: context-dependent findings · transportability · ecological validity