DBox: Scaffolding Algorithmic Programming Learning through Learner-LLM Co-Decomposition

Research Background and Problem Statement

• Identified Problems or Challenges:

  • Current programming learning tools (e.g., LeetCode, ChatGPT) often provide overly comprehensive or generic assistance, which can lead learners to rely on complete solutions without developing independent problem-solving and critical decomposition skills.
  • Programming learning tools lack alignment with learners' individual problem-solving strategies, as existing tools often fail to adapt to learners' unique approaches.
  • Learners require clear, structured support to organize the decomposition process of complex problems. However, traditional tools lack visualization and segmentation features, making it difficult to foster critical thinking.
  • Many support tools fail to provide fine-grained, real-time evaluation and feedback on learners' problem decomposition processes, making it challenging for learners to track their progress in decomposition and code implementation.

• Why This Is Important:

  • Decomposition skills are a core competency in algorithmic programming, enabling the breakdown of complex problems into manageable components, which is a fundamental aspect of cultivating computational thinking.
  • Over-reliance on answers or hints provided by current tools hinders students' ability to develop independent problem-solving skills, negatively impacting learning outcomes.
  • Developing structured algorithmic thinking is not only crucial for computer science learners but also has broad applications in problem-solving across related fields.

• Research Motivation and Related Work:

  • While large language models (LLMs) currently excel in tasks such as code generation and automated suggestions, they are not specifically designed for educational contexts and may foster dependency.
  • Learning strategies like Parsons problems and stepwise planning have proven effective in enhancing learners' understanding but have yet to fully leverage LLMs' potential for dynamic feedback and personalized support.
  • This study aims to explore how LLMs can be utilized to design interactive tools that support personalized algorithmic programming learning while maintaining learners' autonomy and critical thinking.

Solution

• Proposed Method or Solution:

  • Designed and implemented an interactive learning tool called "DBox" (Decomposition Box), which employs a "learner-LLM collaborative decomposition" approach to guide learners in breaking down algorithmic problems into progressively structured step trees.
  • DBox provides support during two key stages of algorithm problem-solving:
    1. Solution Construction Phase: Assists learners in building a cognitive framework (step tree).
    2. Code Implementation Phase: Ensures alignment, consistency, and correctness between the code and the step tree.
  • The tool dynamically provides real-time feedback and suggestions based on learners' inputs (code or natural language steps) while maintaining learners' control over the decomposition process.

• Innovative Features:

  • Integrates two input modes—code writing and natural language descriptions—and automatically generates step trees from incomplete code to promote structured cognitive modeling.
  • Employs progressive, multi-level hints and fine-grained real-time evaluations to achieve appropriate intervention without excessive interference.
  • Establishes a "learner-led, LLM-supported" collaborative decomposition model, empowering learners with independence and initiative while leveraging LLM feedback for moderate support.

• Implementation Steps and Key Technologies:

  1. Input Modes and Step Tree Construction:
     • Supports learners in describing decomposition steps using code or natural language.
     • Provides a "from editor to step tree" feature that maps code into step trees.
  2. Automated Step Tree Evaluation:
     • Classifies step tree node statuses as correct, incorrect, missing, or requiring further decomposition, providing real-time feedback.
     • Retains learners' original input to highlight personalized learning.
  3. Advanced Hint System:
     • Offers three levels of progressive hints: "general hints," "detailed hints," and "revealing sub-steps," gradually increasing the level of assistance.
  4. Code and Step Tree Alignment Verification:
     • Allows learners to convert step trees into code annotations and verify their implementation progress using a "check alignment" button.
  5. Technical Implementation:
     • Utilizes the GPT-4 model and carefully designed prompt engineering to support various tasks (e.g., step tree generation, error detection, multi-level hint generation).

Research Outcomes

• Specific Results:

  • DBox significantly improved learners' test scores on similar problems, enhancing their algorithmic thinking and independent problem-solving abilities.
  • User experience evaluations indicated that DBox fostered higher cognitive engagement, critical thinking, and a sense of accomplishment among learners.
  • Learners widely regarded DBox as a moderately but effectively supportive tool that provided an appropriate level of assistance.
  • Technical evaluations demonstrated that DBox achieved high accuracy in assessing learners' decomposition steps in code input scenarios.

• Advantages Over Existing Solutions:

  • Compared to tools like ChatGPT, DBox excels in avoiding "simple answer copying" by prioritizing support for learners in constructing their own solution pathways.
  • Unlike the linear interaction interfaces of existing systems, DBox introduces an understandable, visualized tree structure that significantly enhances support for hierarchical decomposition of complex problems.
  • Maintains a focus on learners' personalized problem-solving styles, providing appropriate assistance within unfiltered cognitive pathways.

• Experimental Results:

  • Learners using DBox achieved significantly higher accuracy scores on test tasks.
  • Self-reported learning gains, confidence in solving similar problems, and a sense of self-efficacy were significantly higher compared to baseline tools.
  • Most learners reported that the tool did not cause additional frustration despite the increased cognitive load.

• Limitations and Future Directions:

  • The current study is limited to two types of algorithmic problems. Further research is needed to analyze its applicability across a broader range of problem domains.
  • Learners experienced only a single learning session; future studies should explore the tool's long-term use and its effectiveness in knowledge transfer.
  • The design requires optimization to mitigate LLM error recognition risks and minimize disruptions to learners' cognitive processes.

Through the design and validation of DBox, this study proposes a novel design concept for leveraging LLMs in programming education, providing significant insights into applications for personalized learning, fostering autonomy, and enhancing decomposition skills.