Proxona: Supporting Creators' Sensemaking and Ideation with LLM-Powered Audience Personas

Research Background and Problem Statement

• Identified Issues and Challenges: Content creators need a deep understanding of their audience, but existing tools (e.g., YouTube Studio) primarily provide quantitative data (e.g., view counts, watch time, demographics) and lack in-depth insights into audience motivations and preferences. Additionally, extracting meaningful audience feedback from a large volume of comments is highly challenging. Research also indicates that creators often require more direct and structured audience feedback during the early stages of content ideation to improve their work, but they frequently find it difficult to obtain such support.
• Importance of the Problem: In the highly competitive creator economy, audience interest and engagement directly influence creators' traffic, revenue, and long-term growth. A deep understanding of the audience not only enhances content quality but also helps creators devise more precise strategies.
• Research Motivation: This study is based on the concept of "audience personas" and aims to fill the gap in existing tools by transforming audience comments into interactive, multidimensional personas, empowering creators to gain better audience insights and improve content ideation.

Proposed Solution

• Proposed Approach: Proxona is a system powered by large language models (LLMs) that transforms audience comments and video metadata into interactive, multidimensional audience personas. These personas are virtual constructs based on real data, enabling creators to engage in simulated conversations with them to gather feedback and support during the early stages of content creation.
• Innovations:
  • Introduces a "dimension-value" framework to systematically and structurally organize audience characteristics.
  • Supports natural language interaction with simulated personas, eliminating the need for real audience participation.
  • Provides deep insights into hidden or overlooked audience traits, complementing existing quantitative analysis tools.
• Implementation Steps:
  1. Data Collection: Scrape video comments and metadata from YouTube channels.
  2. Feature Extraction: Use LLMs to summarize audience comments and extract key dimensions (e.g., interests, motivations) and corresponding values (e.g., "visual learner").
  3. Clustering Analysis: Group comments based on the "dimension-value" framework to generate audience personas.
  4. Persona Generation: Use LLMs to create detailed persona profiles, including occupation, motivations, video preferences, etc.
  5. Interactive Features: Enable natural language conversations with personas and provide personalized feedback based on draft content.

Research Outcomes

• Specific Results:
  • The dimensions and values of system-generated audience personas demonstrated high relevance in evaluations (average relevance score close to 4.0/5) and good distinctiveness between personas (overlap rate below 7%).
  • Technical assessments showed that Proxona generated more diverse and multidimensional personas compared to baseline methods directly using LLMs.
  • The system provided YouTube creators with clear audience insights, enabling them to confidently develop content strategies.
• Advantages and Application Examples:
  • Superior to Existing Tools: Not only provides macro-level quantitative data but also extracts deep, contextualized audience traits from user comments.
  • User Feedback: Creators using Proxona gave high ratings, noting that it helped them identify hidden audience groups, such as "eco-friendly bloggers" or "novice gardening enthusiasts."
  • Creative Support: Creators enriched their content logic using persona feedback and collaborated with virtual audiences to address creative challenges.
• Experimental or Evaluation Results:
  • In user experiments, Proxona significantly improved the satisfaction of 11 creators in content ideation and audience exploration compared to baseline methods (p < 0.05).
  • Users found conversations with Proxona's simulated personas to be highly consistent (scores > 6.3) and reflective of specific audience perspectives.
  • The system's hallucination rate (i.e., probability of generating false information) was as low as 5%, ensuring the credibility and reliability of feedback.
• Limitations and Future Directions:
  • Currently relies primarily on comments as the data source, potentially overlooking non-commenting audience groups.
  • Performance in supporting multiple languages needs optimization, such as improving the model's handling of non-English content like Korean.
  • In cold-start scenarios (e.g., new channels), limited data may constrain Proxona's effectiveness. Future work could explore simulating personas based on data from similar channels.
  • Creators may develop excessive reliance on system feedback or misunderstand its capabilities, necessitating enhanced transparency and user education in future research.