Lost in Translation: The Value of Verbalizations in Interpreting 3D Computer-Aided Design Workflows

Paper Title

Lost in Translation: The Value of Verbalizations in Interpreting 3D Computer-Aided Design Workflows

Publication Info

• Topic area: AI-assisted tools for 3D CAD workflows, focusing on leveraging verbalizations for context-aware support.
• Keywords: Think-aloud computing, CAD workflows, AI assistance, design intent, verbalizations, feedback quality, context-aware systems, generative AI, human-AI interaction, design rationale.

Background and Problem

• Problem / challenge: Current AI assistance for CAD lacks access to rich design intent, relying only on geometric and command history data, which fails to capture designers' goals and rationale.
• Significance: Capturing design rationale is critical for improving AI's ability to provide meaningful, in-situ feedback, enhancing productivity and design quality in CAD workflows.
• Motivation and related work: Existing research has explored CAD task support through generative AI, but the broader potential for AI to assist the entire CAD workflow remains underexplored. Prior studies have shown the value of think-aloud protocols for documentation but have not investigated their use for enabling AI-driven feedback.

Solution

• Proposed approach: Think-Aloud Computing for CAD workflows, leveraging designers' verbalizations to surface design rationale and inform context-aware AI assistance.
• Novelty:
  1. A multi-phase study evaluating think-aloud verbalizations from CAD designers and their impact on expert feedback quality.
  2. A qualitative analysis of the types of information verbalized by designers and their value for interpreting workflows.
  3. Insights into effective feedback characteristics and how verbalizations improve alignment with designers' goals.
  4. Design considerations for AI systems to scaffold verbalizations and provide contextually-aware support.
• Procedure and key techniques:
  • Conducted a three-part study:
    1. Recorded think-aloud verbalizations from 10 designers during CAD tasks.
    2. Had 10 experts review recordings under two conditions (with/without think-aloud audio) and provide feedback.
    3. Interviewed designers to evaluate feedback relevance and helpfulness.
  • Analyzed verbalizations and expert feedback using thematic coding and hybrid qualitative methods.

Results

• Concrete findings:
  • Designers' verbalizations fell into six categories: process (35.5%), design rationale (29.4%), challenges (14.5%), reflection (12.3%), to-do (2.2%), and other (6.2%).
  • Think-aloud data improved feedback relevance and helpfulness, particularly for product-level considerations (62% helpful with think-aloud vs. 28% without for Tcreate tasks).
  • Experts valued verbalizations for revealing goals, rationale, and challenges, which were absent in silent recordings.
• Advantage over baselines:
  • Feedback with think-aloud data was rated higher in relevance, helpfulness, actionability, and potential impact on the design process compared to feedback without verbalizations.
• Experiments / evaluation:
  • Study involved 10 designers and 10 experts, with recordings analyzed across two conditions (Think-Aloud and No-Think-Aloud).
  • Feedback was evaluated by designers through structured ratings and semi-structured interviews.
• Limitations and future work:
  • Feedback quality was assessed subjectively by designers, and real-time feedback was not tested.
  • Study focused on novice to intermediate designers; future work should explore expert-level needs.
  • Plans include developing a think-aloud computing prototype for in-situ feedback and investigating multimodal AI interactions.

Summary

This paper investigates the potential of think-aloud verbalizations to enhance AI assistance in 3D CAD workflows. Through a three-part study, the authors demonstrate that verbalizations reveal critical design rationale and challenges, enabling more relevant and helpful feedback from experts. Designers preferred feedback sessions informed by think-aloud data, highlighting its value for context-aware support. The findings suggest opportunities for AI systems to scaffold verbalizations, interpret nuanced cues, and provide multimodal feedback. This work lays the foundation for developing intelligent, contextually-aware AI assistants for CAD, with implications for collaborative design and knowledge sharing.