PrivateTalk: Activating Voice Input with Hand-On-Mouth Gesture Detected by Bluetooth Earphones

We introduce PrivateTalk, an on-body interaction technique to activate voice input with the Hand-On-Mouth gesture. When "talking" to a computing device (e.g., a smartphone), a user covers his/her mouth with the hand from one side. PrivateTalk provides two essential benefits for voice input simultaneously. First, the Hand-On-Mouth gesture protects talking privacy by lowering down the voice spreading out and shielding the mouth and lip movements from seen by people around. Second, it uses an on-body hand gesture which removes the need for Wake-Up-Word and is more accessible than a physical/software button when the device is not in hand. We propose a novel sensing technique to recognize this asymmetric Hand-On-Mouth gesture with Bluetooth earphones (e.g., AirPod). When the mouth is covered from one side, the amplitude and Mel-frequency cepstral coefficients (MFCC) of the two audio signals from earphone's microphones are significantly different. This feature is stable and robust across users, and we leverage it to recognize the Hand-On-Mouth gesture. Evaluation results show that the recognition accuracy is as high as 98.33\%. Users' subjective feedback indicates that PrivateTalk is intuitive and consistently appreciated by users.