XSynth: GenAI-Empowered Shared Mental Model Building for Conceptual Design Collaboration in Extended Reality

Paper Title

XSynth: GenAI-Empowered Shared Mental Model Building for Conceptual Design Collaboration in Extended Reality

Publication Info

• Topic area: Human–Computer Interaction (HCI) and Computer-Supported Cooperative Work (CSCW) in XR-mediated design collaboration.
• Keywords: Shared mental models, conceptual design, extended reality, generative AI, knowledge graphs, C–K Theory, team cognition, design collaboration, creativity support, XR tools.

Background and Problem

• Problem / challenge: XR-mediated design collaboration tools lack structured cognitive scaffolds to support shared mental model (SMM) building, leading to fragmented ideation, misaligned mental models, and reduced team performance.
• Significance: Addressing cognitive challenges in XR-mediated design collaboration is critical for improving team alignment, creativity, and design outcomes in interdisciplinary and immersive settings.
• Motivation and related work: Prior studies have explored XR’s potential for enhancing co-presence and creativity but have not systematically addressed cognitive scaffolding for SMM building. Concept–Knowledge (C–K) Theory offers a structured framework for design reasoning, but its application in XR-mediated collaborative tools remains unexplored.

Solution

• Proposed approach: XSynth, a GenAI-powered XR system grounded in C–K Theory, uses knowledge graphs to scaffold individual and shared mental model building during conceptual design collaboration.
• Novelty:
  1. Integration of C–K Theory with GenAI to structure and visualize team cognition in XR.
  2. Development of a knowledge graph-based approach for externalizing and aligning individual and shared mental models.
  3. Empirical validation of XSynth’s effectiveness in reducing cognitive workload, enhancing creativity, and improving design outcomes.
• Procedure and key techniques:
  • XSynth operationalizes a design-cognition co-evolution pathway: design problem → individual mental models (IMMs) → shared mental model (SMM) → design solution.
  • GenAI scaffolds cognition using prompt engineering based on C–K Theory and the F–A–T schema (Function, Appearance, Technology).
  • Knowledge graphs externalize IMMs and synthesize them into unified SMMs, enabling visualization and interaction in XR.
  • Technical implementation includes mental model acquisition, structuring, merging, and visualization using ByteDance’s Doubao LLM, NetworkX, and Spatial XR platform.

Results

• Concrete findings:
  • XSynth reduced mental demand (53.70 vs. 67.07) and improved perceived performance (50.13 vs. 65.77) in NASA-TLX workload scores.
  • Creativity Support Index (CSI) scores showed significant improvements in exploration (6.78 vs. 5.77), expressiveness (6.73 vs. 5.53), immersion (7.33 vs. 6.07), and total score (66.48 vs. 58.84).
  • 5-PSMMS scores for perceived SMMs were significantly higher across execution (5.43 vs. 4.69), interaction (5.99 vs. 4.84), composition (5.58 vs. 4.67), and temporal coordination (5.90 vs. 4.67).
  • Expert evaluations rated design concepts produced with XSynth higher in novelty (6.80 vs. 6.17), completeness (7.03 vs. 5.87), and quality (6.87 vs. 6.03).
• Advantage over baselines: XSynth outperformed a baseline system (Doubao LLM without theory-grounded scaffolding) in reducing cognitive workload, enhancing creativity support, strengthening perceived SMMs, and improving design quality.
• Experiments / evaluation:
  • A 2×2 within-subject counterbalanced experiment with 30 participants (10 teams) compared XSynth and baseline systems in two conceptual design tasks.
  • Metrics included NASA-TLX, CSI, 5-PSMMS, TEQ, and expert evaluations of design quality.
• Limitations and future work:
  • XSynth relies on text-based inputs, limiting representational richness; future work should explore multimodal inputs like sketches.
  • Stability issues in knowledge graph generation require technical refinement.
  • Participants were predominantly senior design students; broader participant pools and real-world applications are needed.
  • Future studies could investigate neurocognitive methods to track dynamic SMM evolution.

Summary

XSynth integrates GenAI and C–K Theory to scaffold shared mental model building in XR-mediated conceptual design collaboration. By externalizing individual and shared cognition as knowledge graphs, XSynth reduces cognitive workload, enhances creativity, and improves team alignment and design outcomes. Empirical findings demonstrate its effectiveness compared to baseline systems, while qualitative insights reveal its socio-emotional and cognitive impacts. This research contributes to HCI and CSCW by offering a scalable framework for GenAI-powered cognition support in immersive collaboration settings.