Felt privacy and actual data flow come apart
Aliases: felt privacy · always-on mental model · architecture mismatch
What it is
A speaker that sits on all day in a home produces a felt privacy model: the lamp is dim so it is not listening, it did not speak so the words are still only in the air, the kitchen is ours so this talk has no second audience. The data flow is a different diagram — a wake-word ring buffer on the chip, cloud recognition after a threshold, possible human review, history tied to an account. The two diagrams routinely fail to match. The question is whether “what it feels like it is doing” and “what actually moved” are the same event. It is not whether an indicator can be switched off, and it is not the social cost of being overheard in public.
Why it happens
Felt privacy is built from socially visible cues: is anyone addressing it, did it speak, is the lamp the quiet kind of dark. Data flow is an invisible state machine: are a few seconds of waveform in the buffer, did a near-wake graze the threshold, which bucket did the audio enter after that. The home enlarges the mismatch — people treat kitchen volume as private, and the far-field microphone’s working area is exactly that kitchen. A dark lamp rules out “currently speaking”; it does not rule out “the ring buffer is still turning.” People also equate “I did not call it” with “it does not have this utterance.” Near-wakes, television dialogue grazing the detector, talk that falls into an still-open session window, all move content while the felt state remains idle.
Mismatch has two directions. Overestimate (every sentence goes to the cloud) produces excessive muting. Underestimate (idle means zero capture) leaves medical, money, and relationship talk undefended. Once the device has become furniture, underestimate is the common error: furniture is not watched.
Studying it
Externalize the mental model. Ask household members to draw where the last sentence went. Card-sort idle, on-device wake spotting, in-recognition, uploaded, under human review. Lay the drawings next to the real architecture; the mismatches are the object of study. Useful interview questions for smart speakers ask “what do you think it is doing,” not “do you care about privacy” — the latter measures attitude, the former measures the data-flow model.
Pair this with a mute diary: each cover, unplug, or turn-the-speaker-away, and whether an after-wake upload actually happened then. If protective acts systematically land when the pipeline was not open, the mismatch is measured. A general privacy-attitude scale measures a different seam (attitude versus behavior) and cannot stand in for the drawing.
Where it stops holding
On-device spotting that never leaves the chip narrows the harm of underestimate, but the ring buffer is still there; people may still map “not processing at all” onto “only sliding past locally.” Readers of architecture notes mismatch less and are not stand-ins for household users. When the error is overestimate, muting needs a different explanation than a loose felt model. In the first days after unboxing, before the object becomes furniture, people watch the lamp and the mismatch briefly shrinks.
Applying it
- Write states as a table a user can match: on-device spotting, post-wake upload, human review each get a row, and each row a perceptible difference (lamp, sound, whether it appears in history). One lamp must not cover three rows.
- Ask people who did not design it to draw idle, a true wake, and a silent near-wake. “Lamp dark means nothing in the cloud” is still misaligned.
- Do not write “we do not listen.” That is felt-privacy language. Write where the buffer stops, where audio goes after wake, and what history keeps.
- How to check: walk three utterances against the architecture (chatter, a call, chatter after the call). Ask everyone in the home whether each sentence left the room. The sentence whose answer disagrees with the architecture is the one whose indicator or default upload still needs work.