Orality: A Semantic Canvas for Externalizing and Clarifying Thoughts with Speech

Paper Title

Orality: A Semantic Canvas for Externalizing and Clarifying Thoughts with Speech

Publication Info

• Topic area: AI-assisted tools for thought clarification using speech and semantic visualization.
• Keywords: Thought externalization, semantic canvas, speech-to-text, large language models, AI-assisted thinking, sensemaking, metacognition, verbal structuring, cognitive scaffolding, interactive visualization.

Background and Problem

• Problem / challenge: Existing speech-to-text tools generate linear, disorganized transcripts that are difficult to review and synthesize. Current AI conversational interfaces, like ChatGPT, struggle with maintaining context and representing non-linear thought processes effectively.
• Significance: Clarifying and organizing thoughts is critical for complex tasks like research planning, decision-making, and creative problem-solving. Tools that better align with human cognitive processes can significantly enhance productivity and insight generation.
• Motivation and related work: Prior tools like digital mind-mapping software and AI-based conversational interfaces provide some support for thought externalization but fail to address the iterative and non-linear nature of human thinking. Recent advancements in LLMs and interactive visualization present an opportunity to bridge this gap.

Solution

• Proposed approach: Orality, an AI-enhanced semantic canvas that transforms speech into a node-link diagram for interactive thought clarification.
• Novelty:
  1. A speech-first workflow that converts linear spoken input into a dynamic, spatial semantic representation.
  2. A four-layer framework for iterative thought clarification, integrating externalization, structuring, elaboration, and reflection.
  3. AI-powered features like verbal restructuring, thought-provoking questions, and conflict detection embedded within the semantic canvas.
  4. Visualization of thought evolution to support metacognitive reflection and iterative refinement.
• Procedure and key techniques:
  • Speech input is processed into semantic nodes and topics using LLMs.
  • A two-stage layout algorithm (PCA-based semantic placement and dynamic refinement) organizes nodes spatially.
  • Users can issue verbal commands to reorganize content, generate AI suggestions, and detect logical conflicts.
  • Thought evolution is visualized through a timeline, and outputs can be exported in various formats.

Results

• Concrete findings:
  • Orality improved post-task thought clarity ratings (M=5.42, SD=0.90) compared to the baseline (M=4.917, SD=0.996).
  • 8 out of 12 participants preferred Orality for supporting their thinking process.
  • Features like "Ask Me Questions" and verbal structuring were highly rated for usefulness.
• Advantage over baselines:
  • Orality supported deeper, iterative, and non-linear thinking compared to the linear, chat-based baseline.
  • Participants reported more active sensemaking and targeted thought development with Orality.
• Experiments / evaluation:
  • A within-subject lab study with 12 participants comparing Orality to a ChatGPT-based baseline.
  • Tasks involved thought clarification on self-proposed topics, with qualitative and quantitative data collected.
  • Metrics included thought clarity ratings, NASA-TLX workload scores, and feature-specific usefulness ratings.
• Limitations and future work:
  • Lab setting may limit ecological validity; real-world applications need exploration.
  • Variability in participant tasks affects comparability.
  • System output instability (e.g., misclassification, incomplete conflict detection) requires technical improvements.

Summary

Orality is an AI-enhanced semantic canvas designed to support thought clarification by transforming speech into a dynamic, interactive node-link diagram. It integrates features for verbal structuring, AI-guided elaboration, and iterative reflection, enabling users to externalize, organize, and refine their thoughts effectively. A user study demonstrated Orality's advantages over a ChatGPT-based baseline, particularly in supporting non-linear, in-depth thinking and active sensemaking. Future work will focus on improving system robustness, exploring diverse semantic representations, and validating the tool in real-world scenarios.