The second honesty check is whether a reader can find the range, population, and exclusions behind the number
Aliases: scope check · filter disclosure check
What it is
The second honesty check is disclosure completeness: can a reader learn from the chart itself (or one click away) which time range, which population or channels, and which filters and exclusions produced this number? This item addresses selective presentation—every number on a chart is a statistic of some filtered population, and undeclared filtering makes the conclusion's scope unjudgeable.
Why it happens
The check's executable form is a "four-element audit": time range (from when to when), population/spatial scope (which channels, regions, segments), filter and exclusion rules (what was included, what was removed, under what rule), and sample size or coverage (how much data the conclusion rests on). All four present on the chart or in an appendix means pass. The check's point is to hand readers the materials for judging "does the filtering serve the conclusion": with the four elements, a reviewer can independently assess whether the filtering is reasonable (does "counting only registered users" hide funnel drop-off?), without relying on the chart maker's self-restraint. The check should also cover default filters—in interactive tools users may be unaware of stacked filters (the same group's scope-disclosure knowledge), so the check includes confirming that currently active filters are visible beside the chart.
Where it stops holding
The disclosure standard grades with the chart's distribution range: internal exploratory charts may satisfy it via the interactive status bar (conditions visible as set), while outward-facing charts must fix the elements on the chart face. The check should not reward redundancy—the four elements fit in one summary line plus hover detail; what is checked is "availability," not "cramming text onto the chart." For multi-source charts, audit the four elements per source (each source's range and definition declared separately).
Applying it
- Check action: verify the four elements (time, population, filter rules, sample size/coverage) are obtainable for each chart; any missing element fails.
- Outward charts require the four elements fixed on the face or in the appendix; interactive charts require live display of active filters beside the chart.
- Record the check result in the publishing checklist, alongside the axis check.
- Verification: have a colleague who did not build the chart answer the four elements from the graphic alone; any unanswerable element means the check fails.
Related
- Same group: U10.04.1 Check whether axes start at zero or are truncated or broken · U10.04.3 Check whether title and annotation claims are supported by the graphic itself · U10.04.4 Check whether uncertainty, missing proportion, and sample size are labeled · U10.04.5 Check whether the conclusion changes under another reasonable rendering · U10.04.6 The checklist runs before publication, not after being challenged
- Nearby: U10.02.3 Data range and filter conditions must be stated · U8.07.2 Data source and update time must accompany the chart
- Search terms:
data scope check·filter disclosure·reporting checklist
Cards in the same group
- U10.04.1The first honesty check is whether the axis starts at zero, with any truncation clearly marked
- U10.04.3The third honesty check is whether the title's claim is actually backed by what the graphic shows
- U10.04.4The fourth honesty check is whether uncertainty, missing data, and sample size are labeled at all
- U10.04.5The fifth honesty check is whether the conclusion survives being redrawn a different, reasonable way
- U10.04.6Run before publication, the honesty checklist prevents distortion; run after a challenge, it only excuses it