What Does AI Do for Cultural Interpretation? A Randomized Experiment on Close Reading Poems with Exposure to AI Interpretation

Paper Title

What Does AI Do for Cultural Interpretation? A Randomized Experiment on Close Reading Poems with Exposure to AI Interpretation

Publication Info

• Topic area: Impact of AI assistance on cultural interpretation and close reading of poetry.
• Keywords: AI assistance, close reading, cultural interpretation, interpretive performance, subjective experience, poetry analysis, human-AI collaboration, interpretive reasoning, generative AI, humanities education.

Background and Problem

• Problem / challenge: The integration of AI into cultural interpretation, particularly close reading, raises concerns about whether AI can enhance interpretive performance without diminishing the pleasure derived from the process. Prior research has focused on AI's role in writing and general reading comprehension but has not adequately addressed its effects on interpretive reasoning for cultural texts.
• Significance: Close reading is a foundational skill for critical thinking, literacy, and cultural engagement. Understanding how AI affects this skill is crucial for designing tools that support human reasoning without undermining the intrinsic rewards of cultural interpretation.
• Motivation and related work: Previous studies have demonstrated AI's potential in simplifying texts and improving reading comprehension but have not explored its impact on interpretive tasks like close reading. There is also limited empirical evidence on how AI affects subjective experiences such as enjoyment, appreciation, and self-efficacy in cultural interpretation.

Solution

• Proposed approach: A randomized controlled experiment to evaluate the effects of AI assistance on close reading of poetry, comparing three conditions: no AI assistance (Control), a single AI interpretation (AI-Single), and multiple AI interpretations (AI-Multiple).
• Novelty:
  1. Empirical evidence on how AI assistance influences both interpretive performance and subjective experience in close reading.
  2. Insights into the trade-offs between performance and pleasure when using AI for cultural interpretation.
  3. Examination of behavioral engagement with AI assistance, including viewing patterns and self-reported use.
• Procedure and key techniques:
  • Participants (n = 400) were randomly assigned to one of three conditions and completed close reading tasks for three poems.
  • Tasks involved identifying stylistic features and explaining their effects, scored on feature identification, interpretation quality, and writing quality.
  • Subjective experience was measured using appreciation, enjoyment, and self-efficacy ratings.
  • Mixed-effects models analyzed the effects of AI assistance, with additional analyses on behavioral engagement and textual overlap.

Results

• Concrete findings:
  • AI-Single improved both interpretive performance (e.g., Interpretation Quality: +0.865 points) and subjective experience (e.g., Enjoyment: +0.969 log-odds).
  • AI-Multiple improved interpretive performance but had no significant effect on subjective experience and reduced self-efficacy for experienced readers.
  • Participants who heavily relied on AI (high textual overlap) achieved higher performance but reported lower subjective experience.
• Advantage over baselines:
  • Both AI-Single and AI-Multiple outperformed the Control in interpretive performance, with AI-Single showing larger effect sizes.
  • AI-Single uniquely enhanced subjective experience, unlike AI-Multiple.
• Experiments / evaluation:
  • Conducted with 400 participants using three curated poems.
  • Metrics included interpretive performance (feature identification, interpretation quality, writing quality) and subjective experience (appreciation, enjoyment, self-efficacy).
  • Behavioral engagement analyzed through viewing patterns, self-reported AI use, and textual overlap.
• Limitations and future work:
  • Limited to pre-generated AI interpretations; future work could explore conversational AI.
  • Focused on lay readers and foundational close reading tasks; further research could examine advanced interpretive stages and other cultural media.
  • Binary classification of expertise based on humanities coursework; more nuanced measures of expertise are needed.

Summary

This study investigates the impact of AI assistance on close reading of poetry, revealing that a single AI interpretation improves both interpretive performance and subjective experience, while multiple interpretations enhance performance but not pleasure. Behavioral analyses highlight a performance-pleasure trade-off, with heavy reliance on AI diminishing intrinsic rewards. The findings suggest that modest AI assistance can support cultural interpretation without undermining personal engagement, emphasizing the need for calibrated AI designs that balance performance gains with experiential value. These insights have implications for AI integration in cultural and educational contexts.