Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture

Document Title

Think-Aloud Computing: Supporting Rich and Low-Effort Knowledge Capture

Document Information

• Topic Area: Human-Computer Interaction and Design of Computational Support Tools
• Keywords: Think-aloud, knowledge capture, documentation, process recording, human-computer interaction, natural language processing, prototype development, experimental evaluation, usability studies

Research Background and Problem Statement

Identified Issues or Challenges

• When users complete tasks using software, much of the information regarding goals, knowledge, and design intentions is not effectively recorded. This information is often lost after task completion, making it difficult for future collaborators or users to understand the rationale behind design decisions.
• Existing traditional documentation methods, such as code comments or slide notes, require additional effort and fail to capture all useful information. These methods are inefficient and of limited value.

Importance

• Insufficient knowledge capture results in redundant work, misunderstanding of design choices, and increased learning curve difficulty.
• Capturing rich information during the design process can enhance collaboration, reduce errors, and ensure efficient continuation of work.

Research Motivation and Related Work

• Inspired by the traditional "Think-Aloud Protocol" (commonly used in usability studies), the authors propose applying this method to everyday computational tasks to capture users' motivations and processes while completing tasks.
• Related research includes design documentation processing, context capture, and real-time voice technologies, but these methods fail to integrate real-time voice, speech classification, and contextual information into a comprehensive system.

Solution

Proposed Method or Solution

• A system named "Think-Aloud Computing" is proposed, utilizing real-time voice recording to capture users' knowledge, goals, and process information during computational tasks. The method is designed to:
  1. Use prompting components to encourage users to describe their thoughts while completing tasks.
  2. Provide tools to classify and contextualize voice input.
  3. Archive captured information for future use.

Innovations

• Extending the "Think-Aloud Protocol" from traditional laboratory usability studies to everyday computational tasks.
• Integrating real-time voice capture, speech content classification, and task context capture to comprehensively support knowledge recording.
• Offering a user interface to guide users in efficiently recording and reviewing information.

Implementation Steps and Key Technologies

1. Encouraging User Expression: Using small visual controls (e.g., a chart with real-time updates) to subtly remind users to share content about design intentions, issues, and task lists.
2. Capture and Contextualization: Employing natural language processing techniques to assist in classifying spoken content and associating it with software operation context information.
3. Displaying Results: Making captured information easily reviewable and usable in future tasks through searchable text transcripts, embedded annotations, and categorized tags.
4. Technical Implementation:
   • Utilizing Microsoft Azure's speech-to-text technology for real-time voice transcription.
   • Providing an interface for users to modify and refine the real-time voice transcription content.

Research Outcomes

Specific Results

• System Prototype Development: Designed and implemented a "Think-Aloud Computing" prototype that supports prompting, context capture, and real-time knowledge classification.
• Experimental Evaluation:
  • Information Quality: The system captured five types of important information that traditional documentation methods failed to record, including problem-solving steps, additional context, design choices and alternatives, checkpoints, and unconscious design decisions.
  • Workload: Despite the increased volume of captured information, users reported that the effort required for "Think-Aloud" was comparable to traditional documentation methods.

Advantages

• Compared to traditional documentation methods, Think-Aloud Computing captures more potentially valuable but easily overlooked knowledge.
• Helps users reduce the learning or switching costs when reviewing or continuing work in the future.
• Seamlessly integrates voice input with task context, minimizing the cost of switching operational modes.

Limitations and Future Directions

• Limitations:
  • Current experiments have not verified the practical utility of captured information for actual consumers (e.g., future colleagues or users).
  • The system heavily relies on the accuracy of speech recognition and classification, and errors may increase users' review costs.
• Future Directions:
  • Incorporate more advanced natural language processing algorithms to enhance speech classification and contextual analysis.
  • Customize capture and presentation methods for different tasks and user scenarios, such as optimizing interaction and integration in collaboration and feedback contexts.
  • Expand beyond computational tasks to other domains, such as teamwork, physical work, or educational tasks, to capture a broader range of task knowledge.