DraftMarks: Enhancing Transparency in Human-AI Co-Writing Through Interactive Skeuomorphic Process Traces

Paper Title

DraftMarks: Enhancing Transparency in Human-AI Co-Writing Through Interactive Skeuomorphic Process Traces

Publication Info

• Topic area: Transparency and interpretability in human-AI collaborative writing.
• Keywords: Human-AI collaboration, transparency, skeuomorphic design, writing tools, process visualization, interpretive judgment, educational technology, co-writing, AI-assisted writing, visual analytics.

Background and Problem

• Problem / challenge: Current tools for understanding AI's role in collaborative writing are limited to external metadata or abstract visualizations, which disrupt the reading flow and fail to provide in-context transparency about human-AI interactions.
• Significance: Transparency in AI-assisted writing is critical for educators, reviewers, and readers to assess human effort, originality, and cognitive engagement, especially in educational and academic contexts.
• Motivation and related work: Prior tools like HaLLMark and DocuViz focus on provenance and summary statistics but lack fine-grained, in-text visualizations of the writing process. There is a gap in tools that reveal iterative refinement, human effort, and AI involvement directly within the text.

Solution

• Proposed approach: DraftMarks, a web-based interactive text viewer that embeds skeuomorphic visual cues directly into AI-assisted text to reveal human-AI collaboration patterns.
• Novelty:
  1. Introduction of skeuomorphic encodings (e.g., masking tape, eraser crumbs) to represent human-AI writing interactions within the text.
  2. A model-view-controller (MVC) architecture that processes interaction data to generate stakeholder-specific visualizations.
  3. Insights from a formative study with teachers, reviewers, and general readers to tailor visualizations to different needs.
  4. Evaluation showing improved comprehension and transparency compared to baseline tools.
• Procedure and key techniques:
  1. Interaction data from human-AI writing sessions is captured using a version-controlled editor state.
  2. Skeuomorphic encodings (e.g., masking tape for AI-generated text, smudge marks for tone transformations) are mapped to interaction data.
  3. A controller module adapts visualizations based on stakeholder needs (e.g., teachers, reviewers, general readers).
  4. Evaluation through a between-subjects study comparing DraftMarks to a baseline interface.

Results

• Concrete findings:
  • DraftMarks improved comprehension accuracy by 50% compared to the baseline (μ = 4.29 vs. μ = 2.86, p < 0.001).
  • High usability score (SUS = 80.5, σ = 12.8), above the 68 threshold for acceptable usability.
  • Moderate intrinsic load (μ = 4.31), low extraneous load (μ = 3.03), and high germane load (μ = 7.42), indicating effective cognitive accessibility.
• Advantage over baselines:
  • Significant improvement in tracking editorial changes and AI contributions (e.g., Question 1 accuracy increased from 2.9% to 51.4%).
  • Higher transparency ratings for confidence in assessment, information sufficiency, and perception of human effort.
• Experiments / evaluation:
  • Participants: 70 professionals (teachers, journalists, copywriters) split into treatment (DraftMarks) and control (baseline) groups.
  • Tasks: Comprehension of a 600-word AI-assisted essay, followed by questionnaires and open-ended feedback.
  • Metrics: Comprehension accuracy, cognitive load, usability (SUS), and transparency ratings.
• Limitations and future work:
  • Limited support for AI training data provenance and citation transparency.
  • Potential information overload from dense visual encodings.
  • Need for configurable visibility settings to balance authorial agency and interpretive transparency.

Summary

DraftMarks introduces skeuomorphic visual encodings to make human-AI collaboration in writing transparent and interpretable. By embedding cues like masking tape and eraser crumbs directly into the text, it helps readers assess human effort and AI involvement. Evaluation shows significant improvements in comprehension and usability compared to baseline tools, with applications in education, peer review, and general readership. Future work will address challenges like information density, AI provenance, and configurable visibility settings for different stakeholders.