Uncertain recognition should fall to the least-privileged identity

What it is

Speaker recognition does not return a name. It returns a set of likelihood scores. Above an upper threshold, bind to a person; below a lower threshold for everyone, treat as a stranger. The gray zone between the two doors is where the product has to decide. Uncertainty should not pick “the closest enrolled user,” and should not keep the previous turn’s account. It should fall to the least-privileged identity: a guest profile, no calendar, no messages, no pay-on-behalf. This is access control under uncertainty. It is not “do not authorize high-sensitivity actions on a voiceprint alone” — that says even a high score is not enough. Here, when the score is not enough, even low-sensitivity personal data stays closed.

Why it happens

Score distributions for enrolled speakers and for non-users overlap. No calibration removes the overlap band; every decision inside it pays either a false accept (open the wrong person) or a false reject (treat the owner as a guest). A convenience policy fails open in the gray zone: bind the last speaker, bind the primary account, bind whoever scored highest. In a family those heuristics are often right — the last turn was Dad, the primary account is the person who bought the speaker, the top score is frequently correct — so everyday successes hide the error until one of them opens someone else’s mail. Least privilege fails closed: personal indexes stay shut; room-level capabilities remain (lights, timers, shared playback). An extra spoken “it’s me” or a short confirm is cheaper than silently opening personal data. The gray zone is not a transient “recognize once more.” It is the interval in which the threshold design admits it cannot name anyone.

Studying it

Plot score histograms for enrolled and non-enrolled speakers, mark the product’s actual thresholds, and count how many sessions land in the gray zone. Run those sessions through three policies (top score, previous turn, guest profile) and compare how often personal data is opened wrongly against how often the owner is locked out of their own calendar. That is a cost-sensitive decision, not equal-error-rate as the only operating point.

In the field, stress pairs whose voices sit close (teenager and parent, siblings), because the overlap band is wider there. Do not prove “the gray zone is rare” only with unenrolled strangers whose voices sit far from the owner.

Where it stops holding

With a single enrolled user and a high threshold, the gray zone mainly becomes “treat as this person or not”; guest profile is still the right closed direction, the owner will just be asked to confirm more often. If an accessibility user cannot complete a second confirm, fail-closed becomes exclusion and needs another already-verified channel — not a revert to fail-open. Enterprise devices may forbid a guest profile; the gray-zone destination then belongs in the deployment policy, not as a silent fall onto the admin account. Sending every gray-zone session to human review is not feasible in a household product.

Applying it

• Use two thresholds. Bind a personal account only above the upper door. Between the doors and below the lower door, both go to guest. Do not treat “closest” as a name.
• Guest can do room-level actions. It cannot read calendars, contacts, messages, payment methods, or “your” recommendations.
• If the owner lands in guest, say hearably “I didn’t catch who, so I’m on guest” and offer a short confirm. Do not quietly fall back to the primary account.
• How to check: push scores into the gray zone (add noise, change distance, use an unenrolled family member) and watch whether calendar and messaging still open. If they open, the policy failed open. If only lights and timers work, it fell to least privilege.

Related

• Same group: M4.08.1 Mismatch between enrollment and use environments lowers hits · M4.08.2 Identity switches need to be hearable
• Nearby: M4.03 Multi-User Recognition · O1.04 Privacy by Default · M4.07 Felt Privacy of Always-On Microphones
• Search terms: least privilege · speaker ID gray zone · fail-closed identity