A conclusion built on 40% complete data deserves far less trust than one built on 99%, but readers can't tell

What it is

A conclusion drawn from 99% complete data and one drawn from 40% complete data should not carry equal trust—but if charts present only the computed result without the coverage underlying it, readers cannot make that distinction. Missing proportion is not "metadata" for the data-quality team alone—it directly determines how much each number can be trusted: means under high missingness may systematically deviate from truth (when missingness is not random), and rankings under low coverage may be driven entirely by missing patterns rather than real differences. Presenting coverage alongside results is the precondition for readers to calibrate trust themselves.

Why it happens

Missing proportion erodes credibility chiefly through non-random missingness: when missingness correlates with the measured quantity (high-value users more inclined to hide income, failed periods harder to collect), conclusions from the observed cases systematically reflect the "data-available" portion, and this bias does not vanish with more data—the mean of 1,000 biased samples does not converge to the true mean. Even under random missingness, the proportion itself determines estimation variance: dropping coverage from 95% to 50% nearly halves the effective sample size and correspondingly widens the confidence interval. What readers need to judge is "how large is the evidence base behind this number," and missing proportion is the direct measure of that base. The visualization implication: coverage should be treated as an attribute of the result, like the confidence interval, rather than a separate quality report—readers should see "what the conclusion is" and "how thick the foundation is" at the same time.

Where it stops holding

There is no universal "safe threshold" for missing proportion: under random missingness, 10% has little effect on most conclusions and 90% invalidates nearly any conclusion; under non-random missingness, even 5% can thoroughly distort a result (if the missing cases concentrate in one critical subgroup). The informational value of coverage depends on whether it relates to the conclusion's fragility: if the conclusion holds under the reasonable assumption of completely random missingness, displaying coverage is mainly transparency; if the conclusion is sensitive to the missing pattern, coverage is the core evidence for judging credibility. In cross-group comparisons, coverage differences may themselves be the finding—a sudden coverage drop in one channel may be the earliest signal of channel trouble, appearing before the metric's values move.

Applying it

• When a chart's data coverage falls below a threshold (e.g., 90%), display the coverage directly ("data coverage: 72%") and give per-group coverage details in the tooltip.
• On comparison charts where coverage differs significantly between groups, annotate "the comparison may be affected by data-completeness differences."
• Maintain a coverage history chart for key metrics and trigger data-quality alerts on coverage drops.
• Verification: pick any number on the dashboard and check whether its underlying data coverage can be found in the interface; if not, transparency is insufficient.

Related

• Same group: U8.04.1 Missing must be visually distinguished from zero · U8.04.2 Lines crossing missing points fabricate continuity
• Nearby: U10.04.4 Check whether uncertainty, missing proportion, and sample size are labeled · U8.04.1 Missing must be visually distinguished from zero
• Search terms: data coverage · missing rate · data quality indicator