To Type or To Speak? The Effect of Input Modality on Text Understanding During Note-taking

Title of the Paper

To type or to speak? The effect of input modality on text understanding during note-taking

Bibliographic Information

• Subject Areas: Human-Computer Interaction, Educational Psychology, Digital Learning Tools
• Keywords: Digital learning, input modality, voice note-taking, note-taking tools, text understanding, metacognition
• Source: CHI Conference on Human Factors in Computing Systems (CHI ’22)
• Publication Date: April 29–May 5, 2022

Research Background and Problem

• Problem and Challenges: Current technologies enable users to take notes using various input modalities such as keyboards, digital ink, and voice. However, the impact of these different modalities on learning outcomes has not been fully explored.
• Significance: Note-taking, as a learning strategy, helps learners improve text comprehension, focus on content, and encode knowledge into long-term memory.
• Research Motivation and Related Work:
  • Typing is faster than handwriting but often leads to transcription-style notes, which can negatively impact learning outcomes.
  • Advantages of voice as an input modality include lower cognitive load, faster recording of fleeting thoughts, and enhanced social presence in expression.
  • The widespread market adoption of voice tools (e.g., Notability) has prompted research into their actual learning benefits.

Solution

• Methods and Innovations:
  • Compare two note-taking input modalities: keyboard and voice, analyzing their effects on learning comprehension, note quality, and learners' metacognitive judgments.
  • Use a mixed-effects model, combining quantitative test results with qualitative interview data to evaluate the impact of input modalities.
• Implementation Steps and Key Techniques:
  • Study Design: 60 participants completed reading and note-taking tasks for two scientific articles, using either keyboard or voice as the input modality.
  • Evaluation metrics included text comprehension (factual and inferential questions), note content (concept units, detail elaboration), and metacognitive accuracy.
  • Google Docs and the Kaizena plugin were used to provide a voice recording environment.
  • Follow-up interviews explored the advantages and disadvantages of voice note-taking.

Research Findings

Experimental Results

• Text Comprehension:
  • Voice note-taking significantly outperformed keyboard input on inferential questions but showed comparable performance on factual questions.
  • Inferential questions required integrating multiple concept units, and voice notes contained more concept units and richer detail elaboration.
• Note Content:
  • Voice input notes included more concept units (average 5.14 compared to 2.36 for keyboard).
  • Voice notes demonstrated higher levels of detail elaboration, incorporating personal experiences or analogies to explain learning content.
• Metacognition:
  • No significant difference was found between the two input modalities in learners' metacognitive accuracy (ability to predict their test performance).

Advantages and Limitations

• Advantages of Voice Notes:
  • Convenient for quickly capturing fleeting thoughts.
  • Encourages deeper thinking and expression, promoting memory encoding through verbalization.
  • Eliminates the need to divert attention to keyboard input, allowing visual focus on learning materials.
  • Enhances collaborative learning, such as emphasizing tone in group projects.
• Limitations of Voice Notes:
  • Privacy concerns, especially in public spaces.
  • Editing challenges, requiring re-recording.
  • Difficulty in quickly browsing and searching note content.
  • Limited support for visual structures and diagrams.
  • User discomfort with listening to their own voice.

Limitations and Future Directions

• Limitations:
  • The experiment used high-difficulty reading materials, making the results applicable to complex texts.
  • Only short-term effects were analyzed; future research should examine long-term impacts.
  • Participants were prohibited from reviewing notes to control variables; the impact of allowing note review warrants further study.
  • Compatibility between voice notes and visual content recording (e.g., charts) remains unexplored.
  • The experimental setting was controlled; results should be validated in real-world educational contexts.
• Future Directions:
  • Investigate the effects on less complex texts.
  • Separate input and output modalities in research, such as combining voice and text.
  • Develop integrated note-taking tools that support visual structures and voice extensions.
  • Expand applications to diverse populations and fields, especially non-engineering learners and novice students.

Conclusion

• Using voice as a note-taking input modality helps learners produce richer, more conceptual notes and facilitates higher-level text comprehension, particularly in inferential learning tasks.
• If the learning goal is to acquire foundational knowledge, keyboard and voice input modalities perform comparably; for deeper conceptual understanding, voice input demonstrates clear advantages.
• Voice note-taking tools are suitable for personal or online digital learning environments, but their use in public spaces requires consideration of auxiliary feature designs.