ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference

Paper Title

ClearFairy: Capturing Creative Workflows through Decision Structuring, In-Situ Questioning, and Rationale Inference

Publication Info

• Topic area: Knowledge capture in creative workflows using AI-assisted tools.
• Keywords: Creative workflows, cognitive decision steps, rationale inference, think-aloud protocol, generative AI, UI/UX design, knowledge sharing, workflow segmentation, Figma plugin, design rationale.

Background and Problem

• Problem / challenge: Existing methods for capturing decision-making in creative workflows often leave rationales incomplete, fail to surface implicit decisions, and can disrupt the creative process.
• Significance: Capturing and documenting decision rationales is crucial for self-reflection, collaboration, knowledge sharing, and training AI tools to support creative workflows.
• Motivation and related work: Prior approaches like retrospective interviews and in-situ self-explanations capture some decision-making information but fail to integrate multimodal signals (e.g., actions, artifacts) effectively. They also impose cognitive burdens or miss implicit decisions. This paper builds on these methods to address these gaps.

Solution

• Proposed approach: ClearFairy, an AI assistant implemented as a Figma plugin, captures cognitive decision steps by linking self-explanations, actions, and artifacts, detects insufficient rationales, and infers missing reasoning through generative AI.
• Novelty:
  1. Introduction of Clear, a workflow segmentation approach that structures workflows into "cognitive decision steps."
  2. Development of ClearFairy, which uses in-situ questioning and rationale inference to capture and clarify decision rationales.
  3. Creation of a dataset of 417 cognitive decision steps for future research.
  4. Demonstration of ClearFairy’s utility in improving rationale capture and supporting generative AI applications.
• Procedure and key techniques:
  1. Segment workflows into cognitive decision steps using Clear, which dynamically links explanations, actions, and artifacts.
  2. Evaluate rationale sufficiency and prompt clarifying questions for weak or empty explanations.
  3. Infer missing rationales using generative AI based on prior user responses.
  4. Document decision steps into a structured format for reuse and analysis.

Results

• Concrete findings:
  • ClearFairy increased strong explanations from 13.9% (baseline) to 83.2%.
  • 85% of inferred rationales were accepted by users (60.8% as-is, 24.2% after minor revisions).
  • Captured decision steps improved alignment in generative AI applications, with professionals preferring AI-inferred actions in 66.7% of cases.
• Advantage over baselines:
  • ClearFairy captured significantly more strong explanations without increasing cognitive burden compared to a think-aloud baseline.
  • The system enabled users to articulate implicit decisions and provided rationale inference to reduce the burden of constant elaboration.
• Experiments / evaluation:
  • User study with 12 interface design professionals performing web design tasks in two conditions (ClearFairy vs. baseline).
  • Technical evaluation of workflow segmentation and rationale evaluation modules using data from 6 designers.
  • Exploratory applications tested next-action prediction and design artifact generation using captured decision steps.
• Limitations and future work:
  • Limited question variety; future work could explore broader question types (e.g., Socratic questioning).
  • Current reliance on verbal explanations; future iterations may focus on action-based inference to reduce user burden.
  • Expansion to support workflows across multiple software platforms and domains.

Summary

ClearFairy is an AI assistant that captures and clarifies decision rationales in creative workflows by structuring them into cognitive decision steps. It uses in-situ questioning and rationale inference to surface implicit decisions and reduce the cognitive burden of knowledge sharing. In a study with 12 professionals, ClearFairy significantly increased the capture of strong explanations and improved alignment in generative AI applications. The system’s ability to document workflows and infer rationales has implications for learning, collaboration, and enhancing AI agents. Future work will focus on expanding its applicability and reducing reliance on verbal explanations.