The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers

Title of the Paper

The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers

Paper Information

• Topic Area: Intersection of Natural Language Processing (NLP) and Human-Computer Interaction (HCI), focusing on the role of language model-based text suggestions in user behavior and text composition.
• Keywords: Text input, language models, text suggestions, deep learning, neural networks, datasets, human-computer interaction, creativity support

Research Background and Issues

• Problems and Challenges:

  • Current neural network-based language models have shown significant improvements in text generation performance, but their impact on user behavior and text outcomes in human-computer interaction systems remains underexplored.
  • Existing studies primarily focus on using single suggestions to improve input efficiency, neglecting support for user creativity and text expression.
  • Users may face a trade-off between "efficiency and creativity" when presented with multiple phrase suggestions.

• Importance:

  • The broader adoption of language model-based text generation tools requires a deeper understanding of user behavior and its impact on the creative process to optimize tool design.
  • Non-native users may benefit more from these suggestions, but relevant research is lacking.

• Research Motivation and Relevance:

  • Investigating the impact of varying numbers of parallel suggestions on native and non-native users to reveal design trade-offs and address existing research gaps.
  • Developing more effective interactive text suggestion systems to support user creativity rather than merely replacing human writing.

Solution

• Proposed Method:

  • Develop a prototype text editor based on the GPT-2 language model, refined for specific business scenarios, to provide users with real-time multi-phrase suggestions.
  • Conduct online experiments under four conditions (0, 1, 3, and 6 parallel suggestions) to analyze user input behavior, creative content, and differences between native and non-native users.

• Innovations:

  • A pioneering study from a UI design perspective on the impact of displaying multiple parallel suggestions.
  • Examining the moderating role of native and non-native language proficiency on suggestion usage, with a focus on non-native user behavior patterns.

• Implementation Steps and Key Technologies:

  • Preliminary Experiment: Recruit 30 participants to explore the prototype and optimize suggestion system parameters.
  • Main Experiment: Use the improved prototype in an online writing task with 156 participants, recording data such as suggestion acceptance, modification, and usage frequency.
  • Data Analysis: Apply Hidden Markov Models (HMM) and Linear Mixed Effects Models (LMM) to uncover behavioral patterns and causal relationships.

Research Results

• Specific Findings:

  • Multiple suggestions better support users in finding suitable phrases, with non-native users exhibiting higher suggestion acceptance rates, gaining more vocabulary from suggestions, and showing lower modification frequency.
  • Increasing the number of suggestions sacrifices efficiency but provides significant advantages in inspiration and creativity support.
  • Identified nine fundamental text input behavior patterns, including focused input, list navigation, and revision.

• Advantages Over Existing Solutions:

  • Compared to single suggestions or word predictions, designs with multiple suggestions offer users more choices, supporting creativity rather than mechanical input.
  • Non-native users benefit significantly from suggestions, providing new directions for personalized design.

• Experimental or Evaluation Results:

  • Experiments revealed that more suggestions significantly increase acceptance rates (e.g., 49% increase with 3 suggestions) and reduce the need for suggestion modifications.
  • Non-native users are more likely to accept suggestions and perceive them as helpful for inspiration and word choice compared to native users.

• Limitations and Future Directions:

  • Suggestion quality still needs improvement, as current designs may introduce biases from model generation.
  • Practical effectiveness may differ from experimental settings; future studies should focus on "long-term" and "real-world" scenarios.
  • Further research is needed on language proficiency, suggestion quality, and design parameters, including adaptive or user-controllable UI designs.
  • Data and models have been made publicly available to support further analysis by other researchers on language preferences, fluency, and interaction behavior.

This paper explores how AI-based language generation models influence user writing behavior and text quality through parallel suggestions, with a particular focus on differences between native and non-native users and design trade-offs. The study establishes a detailed interaction behavior model, supporting comprehensive analysis of such AI tools from efficiency to creativity support.