Partnering with Generative AI: Experimental Evaluation of Model-Led and Human-Led Interaction in Human-AI Co-Creation

Paper Title

Partnering with Generative AI: Experimental Evaluation of Model-Led and Human-Led Interaction in Human-AI Co-Creation

Publication Info

• Topic area: Human-AI interaction for creativity and co-creation
• Keywords: Generative AI, human-AI collaboration, creativity support tools, interaction design, idea quality, idea diversity, perceived ownership, cognitive workload, iterative refinement, co-creation frameworks

Background and Problem

• Problem / challenge: The role of interaction design in human-AI co-creation is poorly understood, particularly how different collaboration modes (human-led vs. model-led) affect creativity outcomes such as idea quality, diversity, and perceived ownership.
• Significance: Understanding these dynamics is critical for designing AI systems that effectively augment human creativity without undermining diversity or user ownership, which are essential for innovation and user satisfaction.
• Motivation and related work: Previous studies have focused on one-shot interactions or static creativity support tools, neglecting iterative and collaborative processes that align with real-world creative tasks. This paper addresses the gap by empirically testing different interaction modes to evaluate their impact on creativity outcomes.

Solution

• Proposed approach: The study evaluates five interaction modes using GPT-4.1: (1) human-led question-mode, (2) human-led suggestion-mode, (3) model-led, (4) vanilla chatbot, and (5) control (no AI support).
• Novelty:
  1. First large-scale experimental comparison (N = 1,126) of model-led and human-led interaction modes in complex creativity tasks.
  2. Identification of human-led question-mode as a strategy to mitigate the quality-diversity trade-off and preserve perceived ownership.
  3. Practical design principles for co-creation systems emphasizing iterative refinement and human engagement.
• Procedure and key techniques:
  • Participants performed creative tasks (e.g., repurposing car features) across five conditions.
  • Creativity outcomes (idea quality, diversity, perceived ownership) were measured using expert ratings, text embeddings, and self-reports.
  • A validation study in a product ideation context confirmed the findings.

Results

• Concrete findings:
  • Idea quality: Significantly higher in question-mode (d = 0.36) and model-led (d = 0.55) compared to control.
  • Idea diversity: Highest in question-mode (d = 0.76 vs. model-led), with diversity increasing after interaction.
  • Perceived ownership: Highest in question-mode (d = 0.57 vs. model-led), comparable to control.
• Advantage over baselines:
  • Question-mode outperformed model-led and vanilla in preserving idea diversity and ownership while maintaining high quality.
  • Model-led improved quality but reduced diversity and ownership.
• Experiments / evaluation:
  • Main study (N = 486): Participants completed an Alternative Uses Test (AUT) task.
  • Validation study (N = 640): Product ideation task for UK university students.
  • Metrics: Expert-rated quality, cosine similarity for diversity, and self-reported ownership.
• Limitations and future work:
  • Generalizability to multimodal tasks (e.g., visual design) is untested.
  • Effects on domain experts and longitudinal impacts remain unexplored.
  • Future work could explore hybrid interaction modes and field studies in real-world settings.

Summary

This study demonstrates that interaction design significantly influences creativity outcomes in human-AI co-creation. The human-led question-mode enhances idea quality, preserves diversity, and maintains perceived ownership, making it a promising framework for creativity support tools. By contrast, model-led approaches improve quality but at the expense of diversity and ownership. These findings highlight the importance of iterative, human-centered interaction designs that stimulate reflection and active engagement. The results are robust across tasks and domains, offering actionable principles for designing effective co-creation systems.