"Same Voice, Different Language": An Exploration of Voice-Cloned Translation to Support Non-Native Speakers in Online Meetings

Best Paper
Multilingual & Cross-Cultural Voice InteractionVoice User Interface (VUI) DesignHuman-LLM CollaborationUI/UX DesignersAI/ML Researchers & EngineersUniversity Professors & Researchers

Cross-lingual meetings have become essential for global collaboration, yet current translation technologies often strip away vocal identity — the unique speaker characteristics that convey nuance and social presence. While generic text-to-speech (TTS) provides basic intelligibility, it creates a disconnect between speakers and their translated voices, potentially undermining engagement and comprehension. This paper investigates whether voice cloning technology can bridge this gap by preserving speaker identity in real-time translation. We present a controlled study comparing four voice conditions in meeting interpretation: original speech, gender-neutral TTS, gender-matched TTS, and voice cloning. Through a within-subjects experiment with 45 participants, we demonstrate that voice cloning significantly reduces mental workload ($p < .001$) and enhances user experience across pragmatic quality ($p < .001$), hedonic quality ($p < .001$), and overall satisfaction ($p < .001$) compared to traditional TTS. While original speech maintained advantages in naturalness, voice cloning achieved superior intelligibility, social impression, and user preference. Qualitative analysis revealed that participants valued voice cloning for preserving speaker identity and improving conversation tracking in multi-speaker scenarios. Our findings suggest that identity-preserving translation represents a significant advancement for cross-lingual communication systems, offering both cognitive and social benefits. We conclude with design implications for integrating voice cloning into meeting platforms while addressing ethical considerations around consent and transparency.

Quick Actions

Share

Share this page

ios_share

https://hci.top/en/papers/iui/226671/2026

AdRecommended

Learn AI Coding at CodeNow

At a Glance

Paper Snapshot

fact_check
dataset
Source
IUI
calendar_month
Year
2026
emoji_events
Award
Best Paper
group
Authors
6 authors
sell
Subtopics
Multilingual & Cross-Cultural Voice Interaction, Voice User Interface (VUI) Design, Human-LLM Collaboration
work
Professions
UI/UX Designers, AI/ML Researchers & Engineers, University Professors & Researchers
article
Content Status
Abstract only
hub
Related Papers
2 related papers