Deletion has to cover derived data
Aliases: speaker embedding residue · voice derivatives · leftover voice profile
What it is
By the time a user taps “delete my recordings,” the pipeline has already grown other objects from those utterances: a speaker embedding, personalized acoustic adaptation, “often played / often bought,” human-review notes, cached intents, a voice profile built from the sound. Derived data is not a copy of the waveform. It is a transformed object that still points at the same person. Until those are covered, deletion does not match “my voice is not with you anymore.” Backup and lineage in general systems are a different protocol. What has to be named here are the derivatives peculiar to a voice pipeline.
Why it happens
Voice derives easily. A few seconds of waveform yield a speaker vector; once that vector sits in a voiceprint store, the original audio is no longer needed to open an account. NLU folds “play jazz” into an interest tag that lives in recommendations. A reviewer’s note stays on a ticket. On-device adaptation writes this person’s tract into a small model on the speaker. After the source audio is gone, these objects can still recognize, still personalize, still be cited in a dispute. The “recordings deleted” the user sees is bound to a filename in object storage, not to the embedding table, the feature store, or the sentence “because you like.” One derivative is irreversibly aggregated: many people’s speech jointly updated a general model, and a single delete usually cannot excavate that gradient — the deletion notice has to separate that honestly, and must not say “removed from the model” for work the system cannot do. What it can do is: disable this person’s embedding, drop the profile built from their voice, stop using their history for recommendations.
Studying it
Inject an utterance with a unique marker (a rare address, a rare drug name), let personalization, voiceprint, and recommendations run, then delete, and look: does the voiceprint still unlock, does recommendation still cite that request, is the review note still there, is on-device adaptation still biased toward this person. The method is a derivative inventory along the voice pipeline, not a hunt for a plaintext marker in the primary store — plaintext finds copies, not embeddings.
Do not take “that history row is gone from the UI” as coverage. The contrast is identity-bound actions that still complete after delete.
Where it stops holding
Gradients mixed into a general model and no longer reasonably linkable to a person are not usually retrained on one household delete; what can be required is to stop further adaptation on this person’s data. Legally retained calls may freeze derivative notes; isolate them rather than pretend they are absent. A user who deletes text and explicitly keeps a voiceprint so the voice profile still works is making a selective keep — the interface must write those as two independent acts. Counting every cache hit as a derivative makes the list unbounded. Cover embeddings, adaptation models, interest tags, and review notes first.
Applying it
- The delete inventory follows the voice pipeline: waveform, transcript, speaker embedding, on-device adaptation, interests / recents, review notes. Each class has three states — delete / disable / expire. Ticking only waveform is not enough.
- After delete, a voiceprint must not keep unlocking a personal account. If a voice profile is still wanted, it must be re-enrolled; old embeddings must not resurrect it.
- Copy must say that a general model will not be retrained on one delete, and must still guarantee that this person’s profile and recommendations no longer use those utterances.
- How to check: after delete, use the same mouth for an identity-bound request (calendar, playlist). If personal data still opens directly, the embedding remains. If recommendations still surface that unique request, the tag remains.
Related
- Same group: M4.10.1 Audio and transcripts are two data objects that need separate handling · M4.10.3 The default retention period outweighs the settings page
- Nearby: O1.07 Right to Erasure and Data Deletion · M4.05 Retention of Voice Data · M4.08 Multi-User Voiceprint and Account Switching
- Search terms:
voice derivatives·speaker embedding deletion·machine unlearning