Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph Annotation

Title of the Paper

Promptiverse: Scalable Generation of Scaffolding Prompts Through Human-AI Hybrid Knowledge Graph Annotation

Paper Information

• Subject Area: Online learning technologies, knowledge graphs, AI educational technologies
• Keywords: scaffolding prompt, knowledge graph, human-AI hybrid annotation, online education, natural language generation, Ausubel's meaningful learning theory, video learning, educational technology, artificial intelligence, concept mapping

Research Background and Problem

• Problem or Challenge:

  • Current online learning courses often adopt a "one-size-fits-all" design, failing to meet the needs of learners with diverse background knowledge.
  • Video-based learning typically provides uniform content explanations, which may leave students with varying background knowledge unable to comprehend certain parts.
  • The authors point out that generating highly diverse and widely adaptable scaffolding prompts requires significant creative effort, making it impractical.

• Significance:

  • Providing generative scaffolding prompts can help students better learn and understand the knowledge points in videos through questions and prompts. These prompts can check learners' comprehension, enhance engagement, and connect new concepts with prior knowledge, thereby improving their mastery and organization of knowledge points.
  • To meet the diverse needs of online learning experiences, such scaffolding prompts can significantly enhance learning outcomes.

• Research Motivation and Related Work:

  • Inspired by educational psychologist Ausubel's meaningful learning theory, the authors aim to organize and utilize knowledge graphs to generate diverse prompts and reduce the creative burden.
  • Related research focuses on the learning effects of scaffolding prompts and the use of algorithms and artificial intelligence for question generation and knowledge structure representation.

Solution

• Method or Solution:

  1. Propose the Promptiverse framework, which uses knowledge graphs to generate multi-turn scaffolding prompts, integrating knowledge organization and learning model design from meaningful learning theory.
  2. Introduce Grannotate, a human-AI co-creation knowledge graph annotation tool that employs hybrid active strategies to reduce manual effort.

• Innovations:

  • The ability of automated scripts to traverse and generate prompts enables Promptiverse to produce structured and diverse prompts on a large scale.
  • Grannotate leverages AI to recommend knowledge entities and relationships, reducing the difficulty of manually creating knowledge graphs.

• Implementation Steps:

  1. Transform lecture content into hierarchical knowledge graphs, using entities and relationships to form knowledge triples.
  2. Based on the knowledge structure, implement a prompt generation mechanism according to different learning models (e.g., superordinate learning, associative learning).
  3. Use the hybrid annotation tool Grannotate to assist annotators in constructing knowledge graphs through AI recommendations.
  4. Utilize Promptiverse to convert concept entities in the knowledge graph into diverse prompts along specific paths and patterns, aiding learners.

Research Outcomes

• Specific Outcomes:

  • The proposed Promptiverse framework and Grannotate tool enable instructors to generate scaffolding prompts at 40 times the quantity of manual design methods while maintaining similar quality to manually designed prompts.
  • The quality of prompts generated through the human-AI hybrid method is comparable to those designed entirely manually.

• Comparative Advantages:

  • Compared to manual creation, Promptiverse significantly improves the efficiency of prompt generation while achieving greater content diversity.
  • The quality of prompts generated through the human-AI hybrid approach surpasses those generated by purely AI-automated methods.

• Experimental or Evaluation Results:

  • Experiments recruited domain-expert instructors to generate prompts through manual design, pure AI generation, and Grannotate-assisted methods.
  • Quantitative evaluations showed that Promptiverse generated a larger number of prompts, while the quality of Grannotate-assisted prompts was comparable to or matched manual prompts.
  • Using the NASA-TLX questionnaire to quantify participants' cognitive workload, results indicated that Grannotate reduced task time requirements and achieved a good balance between quantity and quality of prompts.

• Limitations and Future Directions:

  • Currently, only triples in the knowledge graph are used to generate prompts, without addressing multi-entity or more complex knowledge relationships.
  • The method primarily targets text-based learning content and does not utilize other media resources to generate prompts.
  • The authors call for extending this research to other disciplines, such as music education and programming instruction, to validate its broader applicability.
  • Future work may integrate pre-trained language models to enable Promptiverse to generate more complex and flexible prompts.

Conclusion

Promptiverse combines Ausubel's educational theory with AI technologies, proposing an innovative prompt generation system that supports diverse teaching. By introducing the Grannotate tool, it significantly enhances the efficiency of prompt creation while balancing quality and content breadth. This research provides a novel approach to improving online learning experiences and holds significant implications and potential applications in the fields of educational science and AI.