A point far from the rest of the data could be a sensor glitch or the most interesting finding in the set

What it is

Points far from the data's body (outliers) come from two entirely different sources: data errors (a faulty sensor logging -999, an extra zero typed at entry, a wrong unit) and genuine but rare values (an anomalous daily sales peak matching a flash-sale event, a user whose spending dwarfs others because they are a wholesaler). The two are visually identical—both are points far from the body—but the correct treatment is opposite: errors should be removed or fixed, while genuine anomalies are precisely the most important signal. Automated "cleaning" of outliers treats both sources alike and may simultaneously discard the error and the most important discovery.

Why it happens

The basis for distinguishing errors from discoveries is not the value itself but assumptions about the data-generating process: a value impossible under the data's physical and business constraints (age 200, negative clicks) is an error; a value that is simply the tail of a distribution (right-skewed income, hub nodes in a degree distribution) is a genuine rarity. The trouble is that many outliers' classification is genuinely ambiguous—an abnormally large order may be fraud (error) or a real whale customer (discovery), and settling it requires business knowledge, not statistical rules. Visualization's role in the judgment is presenting the distribution's shape to the judge: on a histogram or scatter plot, an isolated point separated from the body by a clear gap is more likely an error (the generating process was interrupted), while a continuously tapering tail is more likely the real distribution—but that is a heuristic, not evidence. Automated outlier-detection algorithms (Z-score, IQR, isolation forests) output only a ranking of "degree of deviation" with no source judgment—using them as a to-review list is sound; using them as automatic deletion criteria discards one class of points or the other.

Studying it

The standard method for studying outlier handling's effect on conclusions is sensitivity analysis: recompute the core conclusion on the same dataset under three treatments (keep, correct, remove), and compare robustness. The dependent variable is how much key statistics (means, correlations, model coefficients) shift across treatments; sensitivity to a single treatment choice signals that the outlier carries disproportionate information. In visualization research, one can measure how readers' attention to and attribution of outliers (error vs signal) are influenced by presentation choices (annotation, truncated axes). A methodological caveat: reporting "conclusions unchanged after removal" requires also reporting the removal criteria, otherwise the sensitivity analysis itself is irreproducible.

Where it stops holding

There is no universal right answer for outlier treatment, but there are universal wrong ones—deleting without thought or keeping without thought. Boundary conditions for judgment include: whether domain knowledge can supply a physical or business explanation for the value (explainable → lean genuine; unexplainable → lean error but verify); the point's leverage on the conclusion (a single point substantially changing a regression slope is high leverage and demands care); and data scale (in large samples a single error's influence dilutes, while in small samples a single outlier can dominate the conclusion entirely). Truncated axes that push outliers out of view are the most common implicit "removal"—the data is not deleted, but the reader's opportunity to see it is.

Applying it

• Keep outliers visible by default; never auto-truncate axes. When focusing on the body is needed, use an inset that magnifies the body while retaining a marker of the outlier's existence.
• Flag automatically detected outliers as "pending review" rather than auto-removing, attaching the deviation measure ("4.2x beyond the IQR upper bound").
• Run sensitivity analysis on high-leverage outliers and present the core conclusion under both treatments side by side.
• Verification: audit one data-cleaning pipeline, list every point it removes, and demand the justification for each; any point without a stated justification is a possibly deleted discovery.

Related

• Same group: U8.05.2 Removal requires stated criteria · U8.05.3 Axis compression hides the body of the distribution
• Nearby: U8.05.2 Removal requires stated criteria · U10.04.4 Check whether uncertainty, missing proportion, and sample size are labeled
• Search terms: outlier detection · data cleaning · robust statistics