CritiqueCrew: Orchestrating Multi-Perspective Conversational Design Critique

Paper Title

CritiqueCrew: Orchestrating Multi-Perspective Conversational Design Critique

Publication Info

• Topic area: AI-assisted design critique and human–AI collaboration in creative workflows.
• Keywords: design critique, multi-perspective feedback, conversational AI, Figma plugin, human–AI collaboration, usability, creative tools, cross-functional design, cognitive load, trust in AI.

Background and Problem

• Problem / challenge: Existing automated design critique tools provide static, one-dimensional feedback (e.g., problem lists) that disrupt creative workflows and fail to support the complex, cross-functional negotiations required in professional design critique.
• Significance: Addressing this gap is critical as designers increasingly act as mediators between user needs, business goals, and engineering constraints, requiring tools that enhance creativity, reduce cognitive load, and foster trust.
• Motivation and related work: Prior tools, such as UICrit and SpecAI, focus on static analysis and single-perspective feedback, lacking the ability to integrate conflicting viewpoints or provide actionable, contextualized solutions. This paper builds on the idea of AI as a collaborative partner, aiming to transform design critique into a dynamic, multi-perspective process.

Solution

• Proposed approach: CritiqueCrew, a Figma-integrated tool that employs multi-perspective orchestration of expert roles (UX, Product Vision, Engineering) to provide conversational critique and interactive remediation.
• Novelty:
  1. Multi-perspective critique model simulating cross-functional teams.
  2. Interactive remediation tools that translate abstract feedback into actionable solutions within the design context.
  3. Structured orchestration of expert roles to surface trade-offs and empower designers as decision-makers.
• Procedure and key techniques:
  1. Designers select a Figma frame and provide contextual inputs (e.g., product goals, brand keywords).
  2. Parallel expert modules (UX, Product Vision, Engineering) generate specialized feedback.
  3. A Lead Coordinator synthesizes feedback into a structured critique agenda, prioritizing issues and surfacing conflicts.
  4. Designers interact with feedback via canvas highlighting, real-time previews, and conversational queries to specific roles.

Results

• Concrete findings:
  • CritiqueCrew improved issue coverage (M = 9.58 vs. 7.96, p = .006) and solution effectiveness (M = 5.22 vs. 4.30, p < .001) compared to a static baseline tool (SpecAI).
  • Multi-perspective critique outperformed a unified expert model in issue coverage (M = 11.08 vs. 9.92, p < .001) and solution effectiveness (M = 5.77 vs. 5.02, p < .001).
• Advantage over baselines:
  • Reduced cognitive load (NASA-TLX: M = 40.57 vs. 51.99, p < .001).
  • Higher usability (SUS: M = 77.29 vs. 55.52, p < .001).
  • Increased trust in AI (TAI: M = 3.76 vs. 2.89, p < .001).
• Experiments / evaluation:
  • Study 1: Comparison with SpecAI (N = 24, within-subjects design).
  • Study 2: Comparison of multi-perspective vs. unified expert models (N = 24, within-subjects design).
  • Metrics: issue coverage, solution effectiveness (expert ratings), usability (SUS), cognitive load (NASA-TLX), trust in AI (TAI), creativity support (CSI).
• Limitations and future work:
  • Limited to single-frame evaluations; future work will expand to multi-screen user flows.
  • Evaluation used pre-defined mockups, not participants’ own designs.
  • Focused on individual designers; future studies will include cross-functional teams.
  • Plans for longitudinal field studies, dynamic role customization, and broader participant diversity.

Summary

CritiqueCrew reimagines AI’s role in design critique, shifting from static problem detection to dynamic, multi-perspective solution co-creation. By orchestrating expert roles (UX, Product Vision, Engineering) and embedding critique directly into the design workflow, it significantly improves issue coverage, solution effectiveness, and user experience compared to static tools. Controlled studies validate that role orchestration and structured dialogue foster trust, reduce cognitive load, and enhance creativity. CritiqueCrew offers a novel paradigm for human–AI collaboration in complex, cross-functional design contexts, with implications for the next generation of creative support tools.