Creating and Evaluating Personas Using Generative AI: A Scoping Review of 81 Articles

Paper Title

Creating and Evaluating Personas Using Generative AI: A Scoping Review of 81 Articles

Publication Info

• Topic area: Application of generative AI in persona development within HCI and related fields.
• Keywords: Generative AI, personas, large language models, user-centered design, evaluation, bias, ethical considerations, HCI, multimodal personas, prompt engineering.

Background and Problem

• Problem / challenge: The integration of generative AI (GenAI) into persona development lacks standardization, evaluation frameworks, and ethical guidelines. There is a risk of biases, reduced human oversight, and methodological inconsistencies.
• Significance: Personas are critical in user-centered design (UCD) for representing user needs, and GenAI offers opportunities for automation and scalability. However, unvalidated and biased personas can lead to flawed design decisions, harming inclusivity and reliability.
• Motivation and related work: Traditional persona development is resource-intensive and limited by manual processes. GenAI, particularly large language models (LLMs), can automate persona creation, but prior research has not systematically synthesized how GenAI is applied, evaluated, or its ethical implications addressed.

Solution

• Proposed approach: A scoping review of 81 articles (2022–2025) to map the use of GenAI in persona development, identify trends, gaps, and ethical considerations, and propose actionable guidelines.
• Novelty:
  1. Comprehensive analysis of GenAI applications across the persona development lifecycle.
  2. Identification of recurring challenges, including bias, lack of evaluation, and reduced human involvement.
  3. Development of practical guidelines for ethical and rigorous GenAI persona development.
  4. Exploration of emerging innovations such as multimodal personas and interactive persona systems.
• Procedure and key techniques:
  1. Literature search across five databases (ACMDL, IEEE Xplore, Web of Science, Scopus, arXiv) using tailored queries.
  2. Screening and inclusion of 81 relevant articles based on predefined criteria.
  3. Thematic coding of articles into categories such as technology usage, evaluation methods, and ethical considerations.
  4. Analysis of trends, gaps, and innovations in GenAI persona research.

Results

• Concrete findings:
  • 86% of studies rely on OpenAI’s GPT models, with limited diversity in LLM usage.
  • 45% of articles lack evaluation frameworks; only 27% use LLMs for evaluation.
  • 56.8% of articles discuss ethical concerns, with bias and representational issues being the most common.
  • 61.7% of studies share resources (personas, code, datasets), showing progress in reproducibility.
• Advantage over baselines:
  • GenAI accelerates persona creation, enables multimodal and interactive personas, and democratizes access to persona development.
  • Innovations include multimodal representations, role-playing prompts, and automated pipelines.
• Experiments / evaluation:
  • Evaluation methods include human-driven assessments (58.6%), computational metrics (20.7%), and benchmarking against real-world data (20.7%).
  • Ethical concerns such as bias, societal harm, and reduced human oversight are prominent.
• Limitations and future work:
  • Limited cross-cultural generalizability and reliance on a single LLM provider.
  • Lack of standardized evaluation protocols and participatory design practices.
  • Future research should focus on bias mitigation, technical improvements, and validation frameworks.

Summary

This scoping review examines the use of generative AI in persona development, identifying trends, gaps, and ethical challenges across 81 articles. While GenAI offers significant advancements in automation, multimodal representations, and resource sharing, the field suffers from methodological inconsistencies, reliance on a single LLM provider, and limited evaluation frameworks. Ethical concerns, including bias and reduced human oversight, are prevalent. The study proposes practical guidelines to address these issues, emphasizing multi-model validation, structured outputs, and human oversight. By addressing these challenges, GenAI personas can evolve into a robust and socially beneficial tool for user-centered design.