Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking

Research Background and Issues

• Identified Problems and Challenges:

  • Large Language Models (LLMs) are significantly transforming human creative processes, but their impact on creativity in unassisted scenarios remains unclear.
  • Concerns exist that reliance on generative AI may weaken individual intrinsic creativity and lead to homogenized thinking, potentially stifling collective innovation.
  • Research on collaborative idea generation largely focuses on short-term effects, neglecting the long-term residual impacts on human cognition.

• Significance:

  • Creativity is a core competency in academic, artistic, and social innovation domains.
  • Understanding how collaboration with AI affects unassisted creative abilities is crucial for designing AI systems that enhance rather than diminish human creative potential.

• Research Motivation and Related Work:

  • Previous studies have primarily focused on quantifying the creative performance of AI tools themselves, with little exploration of how these tools influence users' independent creativity.
  • Academic discussions suggest that AI usage may have varying impacts on human creativity (including divergent and convergent thinking), but systematic experimental evidence is lacking.

Proposed Solution

• Proposed Approach:

  • This study investigates the effects of LLMs on divergent thinking (the ability to generate multiple ideas) and convergent thinking (the ability to select and refine ideas) through two large-scale randomized controlled experiments.
  • The experimental design includes three conditions: unassisted (control group), direct answer support (standard LLM), and guided (coach-style LLM).

• Innovations:

  • Focuses not only on task performance under AI assistance but also explores the long-term residual effects of AI on unassisted task phases.
  • Incorporates both divergent and convergent thinking into the research framework to comprehensively examine the role of LLMs in human creative cognition.
  • Simulates multi-round LLM usage in real-world interactive scenarios and analyzes changes in long-term creativity.

• Implementation Steps and Key Techniques:

  • Experiment 1 (Divergent Thinking):
    1. Uses the "Alternative Uses Test (AUT)" to evaluate participants' ability to generate multiple unique ideas.
    2. Participants are assigned to one of three conditions, completing three rounds of AI-assisted exposure tasks (exposure phase) and one round of unassisted tasks (test phase), with metrics recorded for originality, fluency, diversity at individual and group levels, etc.
  • Experiment 2 (Convergent Thinking):
    1. Uses the "Remote Associates Test (RAT)" to assess participants' ability to generate correct answers based on given clues.
    2. Similarly divided into three conditions, participants complete three AI-assisted exposure tasks and two unassisted test tasks within a consistent time frame, with accuracy and subjective perception metrics evaluated.
  • Employs automated scoring techniques, large language models (GPT-4), and semantic embedding tools (e.g., SBERT) to quantify performance and diversity.

Research Findings

• Specific Experimental Results:

  • During the assisted phase, LLM assistance significantly improved task performance, but performance declined in the unassisted phase, especially under the guided LLM condition.
  • In the divergent thinking experiment, groups receiving LLM assistance demonstrated significantly lower originality and creative diversity in the test phase compared to the control group.
  • The convergent thinking experiment revealed that coach-style LLMs might increase participants' cognitive load, resulting in lower accuracy during unassisted tests.

• Comparison with Existing Solutions:

  • Unlike other studies focused on improving AI generative performance, this research emphasizes the residual effects of human-AI collaboration on independent human creativity.
  • Results indicate that while LLMs expand participants' creative output in the short term, they may foster cognitive dependency and homogenized thinking, persisting after task completion.

• Experimental or Evaluation Results:

  • Divergent Thinking:
    • LLM-generated frameworks (e.g., strategy lists) further reduced creative diversity at the group level, exhibiting a "homogenization effect."
  • Convergent Thinking:
    • Compared to direct answers, guided LLM assistance was less effective, possibly due to the cognitive burden of additional information.
  • Subjective surveys showed that many participants rated their creative abilities lower after using AI, particularly under guided support conditions.

• Limitations and Future Directions:

  • The task settings (e.g., AUT and RAT) in this study are relatively short-term and static, making it difficult to fully simulate long-term interactions in real-world work environments.
  • The study does not address multimodal or non-text creative tasks, such as visual arts and dynamic image generation.
  • Future research could focus on the impact of AI on long-term creative processes in dynamic interactions and open-ended scenarios, as well as explore how to design LLM systems that effectively enhance users' independent thinking abilities.

Conclusion

This study systematically reveals the potential inhibitory effects of LLMs on human independent creativity through experimental methods, highlighting the importance of developing AI tools that promote long-term innovation while avoiding "homogenization." It provides critical references for the future design of improved human-AI collaborative creativity support systems and serves as a reminder to remain vigilant about the cultural and cognitive impacts of AI technologies.