TaleBrush: Sketching Stories with Generative Pretrained Language Models

Title of the Paper

TaleBrush: Sketching Stories with Generative Pretrained Language Models

Paper Information

• Research Area: Human-AI Collaboration and Controlled Text Generation
• Keywords: Story Writing, Sketch Interaction, Creativity Support Tools, Story Generation, Controllable Generation

Research Background and Problems

• Identified Problems or Challenges:
  1. Existing story creation tools based on generative language models (e.g., GPT-3) primarily rely on iterative co-creation between users and AI but lack intuitive interaction methods to control the story generation process.
  2. Algorithms for controlled generation face limitations in granularity, as most existing algorithms or tools only provide control at the story level, making step-by-step control difficult.
  3. Interaction methods like text input or sliders are not intuitive for presenting and manipulating generated content.
• Significance: Story creation requires guiding AI to generate content that aligns with user intent while enabling efficient and innovative writing. Providing creators with an intuitive and user-friendly method can significantly enhance the efficiency and effectiveness of human-AI collaboration.
• Motivation and Related Work:
  1. Inspired by Kurt Vonnegut's approach of representing character fortune changes through simple temporal lines, the study aims to develop a more visual and interactive method.
  2. The focus is on designing an interaction method that is both intuitive and powerful enough to assist humans and AI in collaboratively generating storylines while enhancing users' understanding and control over the generated results.

Solution

• Proposed Method or Solution: A system called TaleBrush is proposed, combining sketch-based interaction with GPT-based language models. It enables users to express and control the evolution of characters' fortunes in a storyline through temporal line sketches.

• Innovations:

  1. Utilizes line sketches, allowing users to intuitively control the "fortune" changes of protagonists by drawing temporal sequences.
  2. Seamlessly integrates with generative language models using a learnable prompt technique (soft prompt), enabling the model to effectively respond to user inputs.
  3. Allows users to iteratively generate sentences and perform visual operations to enhance the controllability and consistency of generated content.
  4. Provides additional control options, such as using the speed of line drawing to convey the precision or ambiguity of content changes.

• Implementation Steps/Key Technologies:

  1. Interface Design: Includes a text editor and a canvas interface, where the canvas is used to draw visual temporal sequences of the story (lines representing the protagonist's fortune changes).
  2. Technical Architecture:
     • Features a generation and recognition module based on GPT language models.
     • Employs soft prompt tuning to learn and control fortune values.
     • Adopts the GeDi (Generative Discriminator) extension method to control text generation with continuous values.
  3. Training and Evaluation:
     • Constructs and annotates a suitable dataset for story fortune.
     • Uses an improved GeDi model to guide GPT in generating text that aligns with control parameters.

Research Outcomes

• Achievements:

  1. TaleBrush demonstrated reliable generation control capabilities in both empirical and user tests, particularly in controlling the evolution of protagonists' fortunes.
  2. TaleBrush optimized sentence coherence while maintaining the creativity (novelty) of generated sentences.
  3. User experiment results showed that sketch-based interaction is not only intuitive but also facilitates iterative refinement of generated stories, helping users create more efficiently.
  4. TaleBrush supports various generation scenarios (e.g., continuous text generation, mid-story completion) and allows for quality improvement through minimal regeneration steps.

• Advantages Over Existing Solutions:

  1. TaleBrush addresses the common "trade-off between control and generation complexity," making generation more controllable without sacrificing textual coherence.
  2. Combines visual interaction with generative models, surpassing current technologies in terms of intuitiveness and efficiency.

• Experimental or Evaluation Results:

  1. Technical evaluations showed that TaleBrush reliably responds to line sketch inputs, with control precision adjustable via parameter tuning.
  2. In automated evaluations, stories generated using TaleBrush excelled in balancing control and sentence coherence.
  3. User studies indicated that the generated text inspires creativity (e.g., novel expressions and storylines) and helps overcome writer's block.

• Limitations and Future Directions:

  1. The length of generated text is currently limited; generating longer texts may require adjustments to control granularity and visualization methods.
  2. The model may exhibit cultural and data biases (e.g., generating more stories with Western cultural contexts).
  3. Alignment between user and model understanding of "fortune" requires further optimization.
  4. Support for additional types of control variables (e.g., secondary character fortunes, scene descriptions, and genre definitions) is needed.