U10.02.2Removing unfavorable data points is falsificationdesign

Erasing the one data point that ruins the story is falsification, not cleanup

Aliases: data deletion · outcome-driven removal

What it is

Deleting a data point because it makes the conclusion look bad (erasing the traffic dip from a service outage, removing the pullback that ruins the line's "steady growth" look) differs in kind from cleaning data under a pre-declared rule (the same group's removal-criteria knowledge): the former screens by "this point is unfavorable to the conclusion," the latter by "this point failed collection." Removal driven by consequence to the conclusion is a form of data falsification—one step from fabricating numbers, because the graphic left behind no longer represents what actually happened. The red line has nothing to do with whether "removal is common" or whether "the point was truly anomalous"; the sole criterion is the screening motive.

Why it happens

Interest-driven removal falsifies because it breaks the graphic's implicit promise: readers assume the plotted points constitute all the data the analysis rests on. After deleting the unfavorable points, the dataset the chart presents and the dataset the analyst actually holds diverge—the chart is no longer a faithful statement about the data but a curated advertisement. Its stealth lies in subtraction: unlike fabricated numbers, it adds nothing that does not exist, and "what was left out" is completely invisible in a single static chart. Detecting it requires external information: longer raw data, cross-reference with known events (why is that famous outage not on the chart?), or cross-period consistency from the same analyst (the point that was in last report has vanished this time). Governance inside organizations rests on two legs: an audit trail for removal decisions (who, when, and on what grounds a point was removed), and a culture separating "presenting unfavorable data" from "interpreting unfavorable data"—data must be presented completely, while business explanations of the unfavorable points (the outage was caused by an external failure) are legitimate context, not grounds for deletion.

Where it stops holding

The red line is motive, so several easily confused cases need distinction: removal under a pre-declared quality rule (collection failures, confirmed entry errors) is not falsification even if the post-removal conclusion looks better—the test is that the rule is independent of the outcome and existed beforehand; annotated presentation (keeping the point but noting "confirmed collection failure here") is fully legitimate, preserving completeness while providing context; removing genuinely "non-target events" (excluding a window clearly marked as test traffic in an A/B analysis) has a sound analytical basis, but the removal's existence should still be marked on the chart. The reverse red line also deserves statement: presenting unfavorable data completely does not mean indulging wrong interpretations of it—show the dip and explain its cause without further editing; that is the entirety of what honest analysis requires.

Applying it

  • Establish an audit trail for data modifications: every deletion or change to collected data records the operator, time, and reason, visible to audit.
  • When event-driven dips or spikes appear in reports, annotate the event ("outage in March caused the dip") rather than deleting the points.
  • Make the team rule explicit: any removal must cite a pre-existing rule; removals without a citable rule are prohibited by default, requiring escalation approval with a full trail.
  • Verification: sample report charts and diff them against raw data tables, checking whether plotted points form a complete subset of the raw data; any missing point without a prior rule backing it is a falsification instance.

Related

  • Same group: U10.02.1 The chosen time window decides the trend's direction · U10.02.3 Data range and filter conditions must be stated
  • Nearby: U8.05.2 Removal requires stated criteria · U10.02.1 The chosen time window decides the trend's direction
  • Search terms: data falsification · selective reporting · research integrity

Cards in the same group

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/handbook/U10.02.2