GatheringSense: AI-Generated Imagery and Embodied Experiences for Understanding Literati Gatherings

Paper Title

GatheringSense: AI-Generated Imagery and Embodied Experiences for Understanding Literati Gatherings

Publication Info

• Topic area: Enhancing cultural understanding through AI and embodied cognition.
• Keywords: AI-generated content, embodied cognition, cultural heritage, literati gatherings, multimodal experiences, immersive technologies, cultural resonance, ritual participation, symbolic interpretation, user-centered design.

Background and Problem

• Problem / challenge: Current cultural applications of generative AI focus on aesthetic reproduction but fail to convey deeper cultural meanings, procedural rules, and social dynamics. Immersive technologies often overlook the embodied nature of cultural practices, limiting their ability to transmit complex cultural phenomena like Chinese literati gatherings.
• Significance: Understanding cultural practices like literati gatherings is vital for preserving intangible cultural heritage and fostering cross-cultural appreciation. Bridging the gap between visual representation and embodied participation can enhance cultural resonance and learning.
• Motivation and related work: Prior research has explored generative AI for cultural visualization and immersive technologies for heritage preservation. However, these approaches are rarely integrated, and their ability to support active cultural participation remains underexplored. This study builds on embodied cognition theory to address this gap.

Solution

• Proposed approach: An AI-driven dual-path framework combining AI-generated multimodal content (symbolic path) and embodied participation (physical path) to support cultural understanding.
• Novelty:
  1. Introduction of a dual-path framework linking visual interpretation and embodied participation.
  2. Mixed-methods study (N = 48) demonstrating the framework’s effectiveness in fostering cultural resonance.
  3. Quantitative and qualitative insights into the interplay between AI-generated content and embodied experiences.
  4. Five transferable design implications for creating cultural experiences in digital heritage contexts.
• Procedure and key techniques:
  1. AI symbolic path: Generate multimodal content (images, videos) in four visual styles (ink-wash, fine-brush, cartoon, oil) using generative AI models.
  2. Embodied path: Design physical activities (e.g., pitch-pot, calligraphy, Go, poetry singing) with culturally inspired props and immersive setups.
  3. Mixed-methods study: Compare forward (AI to embodied) and reverse (embodied to AI) sequences, using measures like cultural resonance, presence, and psychological closeness.

Results

• Concrete findings:
  • AI-generated content improved cultural symbol readability (Film-IEQ scores: images M = 5.28, videos M = 5.04).
  • Cultural resonance increased significantly after embodied participation (CR_post: M = 5.54, up from M = 4.73 after videos).
  • Psychological closeness to literati gatherings (IOS scores) rose from M = 3.00 (pre-test) to M = 4.66 (post-test).
  • Ink-wash and fine-brush styles were preferred for their cultural authenticity and lower cognitive load.
• Advantage over baselines:
  • Embodied experiences provided stronger social presence and deeper cultural understanding than AI content alone.
  • The combination of AI and embodiment outperformed either path in isolation, with complementary strengths in symbolic clarity and experiential depth.
• Experiments / evaluation:
  • Participants (N = 48) divided into four groups: Chinese adults (forward/reverse order), cross-cultural adults, and children.
  • Measures included Film-IEQ, CR, ITC-SOPI, IOS, SAM, and KANO analysis.
  • Activities were evaluated in a lab setting simulating a literati gathering with immersive props and media.
• Limitations and future work:
  • Small sample sizes for cross-cultural and child groups limit generalizability.
  • Indoor prototype lacked full historical reconstruction; future work could explore VR-based implementations.
  • Current AI models struggled with physical plausibility and temporal coherence; custom datasets and pipelines are needed.

Summary

This study introduces an AI-driven dual-path framework for cultural understanding, integrating AI-generated multimodal content with embodied participation. The framework was instantiated through GatheringSense, focusing on Chinese literati gatherings. Results showed that AI content clarified cultural symbols, while embodied experiences deepened cultural resonance and social presence. Preferred visual styles (ink-wash, fine-brush) aligned with cultural authenticity, and the combination of both paths significantly enhanced psychological closeness to the cultural practice. The study provides actionable design implications for creating immersive cultural experiences and highlights the potential of AI and embodiment to transform cultural heritage transmission.