Giving Social Media Post Authors More Control over the Translation of their Posts Enhances their User Experience

Several social networking sites offer automated machine translation of posts, but authors usually have no access to view or modify these translations. This may increase authors' concerns about whether the translations convey their intended meaning. To address this issue, we test a theory-driven model about human-in-the-loop translation using an online between-subjects experiment (N = 216). In our study, participants write fictitious social media status updates in a language other than English, which are then translated with one of three levels of automation: machine-provided translation with no author modifiability (MPT), where authors can view the machine translation but cannot modify it; author-provided translation (APT), where authors manually write the translation with no machine assistance; and machine-provided translation with author modifiability (MPT-AM), where authors can edit the machine translation of their post. We collect objective and subjective measures of users' experience using the assigned translation feature. The results from a structural equation modeling (SEM) analysis demonstrate that, compared to the MPT and APT conditions, participants in the MPT-AM condition reported higher measures of perceived control, ease of use, and perceived comfort, which in turn predicted higher perceived system effectiveness and ultimately increased intention to use the MPT-AM translation feature. Follow-up interviews (n = 15) found that participants appreciate the ability to edit and/or remove translations from their posts, even if they did not always do so.