Are Semantic Networks Associated with Idea Originality in Artificial Creativity? A Comparison with Human Agents

Paper Title

Are Semantic Networks Associated with Idea Originality in Artificial Creativity? A Comparison with Human Agents

Publication Info

• Topic area: Artificial creativity and its comparison with human creativity.
• Keywords: Artificial creativity, semantic networks, idea originality, ChatGPT-4o, human creativity, divergent thinking, creativity support tools, cognitive processes, large language models, HCI.

Background and Problem

• Problem / challenge: The mechanisms underlying artificial creativity remain unclear, particularly the relationship between semantic network structures and idea originality in Large Language Models (LLMs). Prior studies have focused on product-oriented definitions of creativity, neglecting the processes involved.
• Significance: Understanding artificial creativity is crucial for designing effective Creativity Support Tools (CSTs) and ensuring meaningful human-machine collaboration in creative tasks.
• Motivation and related work: Previous research comparing human and machine creativity has yielded contradictory results, with machines sometimes outperforming humans in divergent thinking tasks. However, these studies often lack methodological rigor and fail to explain the processes behind creative outputs. This paper builds on psychological theories of creativity and semantic memory networks to investigate artificial creativity systematically.

Solution

• Proposed approach: The study examines the association between semantic network flexibility and idea originality in ChatGPT-4o compared to higher and lower creative humans.
• Novelty:
  1. First empirical evidence linking semantic networks and idea originality in artificial creativity.
  2. Comparison of semantic network structures between ChatGPT-4o and human participants.
  3. Methodological framework for studying artificial creativity using psychological constructs.
• Procedure and key techniques:
  • Human sample: 81 psychology students divided into higher creative humans (HCH) and lower creative humans (LCH) based on median originality scores.
  • Machine sample: ChatGPT-4o responses collected via its chat interface.
  • Tasks: Verbal Fluency Test, Free Association Task, and Alternate Uses Task (AUT).
  • Semantic networks constructed using graph theory and analyzed for structural (ASPL, CC, Q) and percolation (R) metrics.
  • Originality assessed through total production and top originality peaks using expert coders.

Results

• Concrete findings:
  • ChatGPT-4o was less original than HCH but more original than LCH in AUT responses.
  • Semantic networks of ChatGPT-4o were more rigid and segregated compared to both human groups.
  • HCH exhibited the most flexible and interconnected semantic networks, followed by LCH, with ChatGPT-4o showing the least flexibility.
• Advantage over baselines:
  • ChatGPT-4o outperformed LCH in idea originality despite having a less flexible semantic network.
  • HCH consistently outperformed ChatGPT-4o in originality and network flexibility.
• Experiments / evaluation:
  • Statistical analyses included ANOVA, linear mixed models, and percolation tests.
  • Metrics: originality scores, semantic network measures (ASPL, CC, Q, R).
  • Results confirmed expected patterns for originality but revealed unexpected rigidity in ChatGPT-4o's semantic networks.
• Limitations and future work:
  • Limited human sample (psychology students) and reliance on a single LLM (ChatGPT-4o).
  • Black-box nature of ChatGPT-4o prevented manipulation of hyperparameters.
  • Future research should explore other creativity measures, ecological tasks, and the effects of model hyperparameters on creative performance.

Summary

This study provides the first empirical evidence linking semantic networks to idea originality in artificial creativity, comparing ChatGPT-4o with higher and lower creative humans. Results show that while ChatGPT-4o is less original than higher creative humans, it outperforms lower creative humans despite its rigid semantic network. The findings highlight the complexity of artificial creativity, suggesting that hyperparameters and motivational factors may compensate for structural limitations. The paper advances methodological and operational insights for studying artificial creativity and designing CSTs, emphasizing the need for further research on diverse models, tasks, and creativity constructs.