Existing-user databases systematically exclude prospective users
Aliases: incumbent user bias · customer-list sampling · survivorship bias
What it is
Incumbent-user sampling bias arises when the sampling frame contains only currently active or retained users. Prospects, pre-registration abandoners, and unretained inactive or churned users then have no chance of selection. Customer databases and research panels do not inherently contain only incumbents; bias depends on their actual inclusion and retention rules. A retained-user frame can explain continued use but not why people never enter or what blocks first contact.
Why it happens
An active-user list reflects both a historical product funnel and a retention rule. Discovery, price, access, and onboarding determine entry; continued activity determines presence in the current frame; invitation and response then add selection. A frame conditioned on continued use combines coverage error with survivorship bias. A customer database that retains churned accounts, or a panel recruiting prospects separately, can cover some otherwise missing stages. Size cannot recover people absent from the frame.
Studying it
Audit the database's inclusion, retention, and deletion rules before building frames for active users, churned users, incomplete starters, and people with the need who have not selected the product. Document which journey stage each frame covers and whom it misses, then compare knowledge, expectations, and barriers. Recruit prospects through the underlying task or need. When only an active-user list is available, restrict the claim explicitly to current active users.
Where it stops holding
An incumbent database is appropriate for advanced use, migration, or long-term relationships. Prospective users are not homogeneous; people without the focal need should not be included merely for novelty. Recontacting churned users raises privacy and burden concerns. Cross-channel samples may differ in incentives and setting, which should not be mistaken for journey-stage effects.
Related
- Same group: Q1.11.1 Quotas cover target segments · Q1.11.3 No-shows require controlled standby recruitment · Q1.11.4 Strict screening lengthens recruitment
- Adjacent: Q1.04 Sampling and representativeness · Q3.12 Funnel and retention analysis
- Search terms:
incumbent-user bias·survivorship bias·prospective user research