Toolset Action Coding System: A Generalized Method for Tool Measurement

Paper Title

Toolset Action Coding System: A Generalized Method for Tool Measurement

Publication Info

• Topic area: Standardized measurement of artist toolsets across traditional and digital media.
• Keywords: artist toolsets, action coding taxonomy, digital media, traditional media, human-computer interaction, sculptural process, creative workflows, artist-centered design, impedance score, media comparison.

Background and Problem

• Problem / challenge: There is no standardized system to measure and compare artist toolsets across traditional and digital media, making it difficult to evaluate how tools impact workflows and creative processes.
• Significance: Understanding and improving artist toolsets can enhance creative workflows, bridge the gap between traditional and digital media, and inform artist-centered design in human-computer interaction.
• Motivation and related work: Previous studies have explored creative practices and tool design but have focused on qualitative insights or isolated media. Quantitative, cross-media measurement systems remain underdeveloped, leaving a gap in understanding how tools affect artists’ workflows and experiences.

Solution

• Proposed approach: The Toolset Action Coding System (TACS), a generalized taxonomy and coding system for measuring artist actions across traditional and digital media.
• Novelty:
  1. Introduction of a hierarchical taxonomy of action units for artists’ workflows.
  2. Definition of the "sculptural process" as a framework for comparing creative workflows across media.
  3. Development of metrics like the Impedance Score to quantify media obstruction.
  4. Application of the taxonomy to diverse datasets, including video data and think-aloud interviews.
• Procedure and key techniques:
  1. Data collection from six artists across nine media types, including synchronized video feeds and think-aloud interviews.
  2. Annotation and coding of artist actions using ATLAS.ti software, creating a hierarchical taxonomy of core and non-core actions.
  3. Analysis of metrics such as manipulation time, navigation time, and Impedance Scores to evaluate workflows.
  4. Comparison of traditional and digital media workflows to identify patterns and challenges.

Results

• Concrete findings:
  • Artists spend significantly more time navigating in digital media (21.55% in 3D digital media) compared to traditional media (5%).
  • Impedance Scores reveal higher obstruction in digital workflows (e.g., 0.651 for digital sculpture vs. 0.253 for traditional sculpture).
  • Digital media is perceived as more efficient but less enjoyable, with traditional media fostering longer flow states and a stronger connection to the medium.
  • Artists using optimized workflows (e.g., hotkeys) demonstrated lower navigation times and higher manipulation times.
• Advantage over baselines: The taxonomy enables granular, cross-media comparisons of workflows, revealing insights into toolset inefficiencies and opportunities for improvement that were previously unmeasured.
• Experiments / evaluation:
  • Dataset: 25 sessions from six artists across nine media types, including clay sculpture, digital illustration, and animation.
  • Metrics: Time spent on core vs. non-core actions, Impedance Scores, and action streaks.
  • Methods: Thematic coding, think-aloud interviews, and quantitative analysis of annotated video data.
• Limitations and future work:
  • Manual coding introduces potential for human error, though mitigated by team reviews and artist validation.
  • The taxonomy may require adaptation for other creative domains (e.g., welding, songwriting).
  • Further research is needed to explore flow states and their relationship to toolset design.

Summary

The Toolset Action Coding System (TACS) provides a standardized framework for measuring artist workflows across traditional and digital media. By defining a taxonomy of action units and introducing metrics like the Impedance Score, the study reveals significant differences in how artists interact with their tools, highlighting inefficiencies in digital media workflows. The findings emphasize the need for artist-centered design to reduce media obstruction and improve creative experiences. Future work aims to expand the taxonomy to additional media and explore automated coding systems to enhance scalability.