Examining the Text-to-Image Community of Practice: Why and How do People Prompt Generative AIs?

Video

Image generation entered the mainstream with the spread of large machine learning (ML) models capable of generating artistic images from text. AI art enthusiasts generated millions of images online within a few months, revealing the future social, economic, and legal consequences of text-to-image (TTI) generation. This work explores the multifaceted sociology and practices behind this yet understudied phenomenon. We aim to understand text-to-image practitioners, their motivations, practice, and the usability challenges they face in order to envision creative and meaningful interactions with this technology. We analyzed two sets of data, an online questionnaire answered by 64 practitioners gathered on social media groups and DiffusionDB, a large dataset of user text prompts sent to the Stable Diffusion generative model. The questionnaire results suggest that TTI generation is a recreational activity from narrow socio-professional groups whose users employ auxiliary techniques across platforms and beyond request-response interactions. Inherent model limitations and finding suitable prompt formulation are their main problems. The analysis of the DiffusionDB dataset informed the creation of a taxonomy for prompt specifiers and a corresponding model capable of recognizing the semantic content of unseen prompts, which further informed how practitioners structure TTI prompt. We finally discuss the design and socio-technical implications of our research for creativity support.