CoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language Programming

Document Title

CoPrompt: Supporting Prompt Sharing and Referring in Collaborative Natural Language Programming

Document Information

• Subject Area: Human-Computer Interaction (HCI), Natural Language Programming, Collaborative Programming
• Keywords: Large Language Models, Collaborative Programming, Prompt Engineering, Natural Language Programming, User Interface Design

Research Background and Problem

• Key Issues:

  1. With the advancement of code generation capabilities driven by Large Language Models (LLMs), natural language programming is becoming an important programming approach. However, in collaborative programming environments, programmers may face significant challenges in prompt engineering, such as difficulty in assessing collaborators' progress, lack of efficient communication methods, and repetitive updates to prompt code.
  2. During collaborative programming, programmers frequently need to switch contexts, understand collaborators' intentions, and ensure that prompt instructions align with collaborators' results, which increases cognitive workload.

• Research Significance:
  This study aims to address efficiency issues in prompt engineering during collaborative natural language programming. It explores new mechanisms to support collaborative prompt engineering, reducing repetitive updates, improving communication efficiency, and helping programmers better understand and utilize collaborators' work outcomes.

Solution

Proposed System: CoPrompt

• Core Mechanisms:

  1. Sharing Mechanism (Sharing): Programmers can easily share intermediate results without interrupting collaborators' workflows.
  2. Linking Mechanism (Linking): Supports programmers in associating prompts with interactive dependencies, automatically updating related code snippets to reduce repetitive modifications.
  3. Referring Mechanism (Referring): Provides a method to reference collaborators' prompt content, assisting in generating more accurate code.
  4. Requesting Mechanism (Requesting): Programmers can initiate requests for collaborators to complete tasks and provide feedback without requiring real-time communication.

• Implementation Steps:

  1. Develop a rich text editor that supports generating prompt blocks and code blocks.
  2. Establish a tree-structured task and prompt manager (Prompt Wiki) to display task hierarchies and track collaboration progress.
  3. Implement a user interface featuring Message Panel, History View, and Explanation View.
  4. Integrate a large language model (GPT-4 API) to support prompt generation and automatic code updates.

• Innovations:

  • Supports various functionalities required for collaborative prompt engineering, from understanding collaborators' work to efficiently utilizing and sharing work.
  • Provides real-time synchronization and asynchronous collaboration capabilities.
  • Reduces communication costs and cognitive workload, enhancing collaboration efficiency.

Research Outcomes

• Experimental Results:

  1. User studies show that CoPrompt significantly reduces the number of repetitive code updates in collaborative programming, saving task completion time compared to traditional editors.
  2. The system helps users understand collaborators' prompt content and progress, significantly lowering cognitive workload and improving work efficiency.
  3. Demonstrates clear advantages in supporting programmers to utilize collaborators' work, such as generating precise code by referencing collaborators' prompts.

• Advantages:

  • Provides a smoother collaborative programming workflow.
  • Supports programmers in simultaneously handling distributed tasks without waiting for collaborators to complete subtasks.
  • Reduces the need for repetitive modifications to code or prompt content through automatic synchronization.

• Limitations and Future Directions:

  1. The current system design and experimental participants primarily target programmers with LLM experience, and the study does not cover users of all technical levels.
  2. CoPrompt is more suited for tasks in the data science domain, and general-purpose programming requires further exploration.
  3. The highly automated design may make some users feel a lack of control over code generation. Future work could optimize the balance between user control and automation.

Conclusion

This paper proposes an innovative tool, CoPrompt, for collaborative natural language programming scenarios, featuring four mechanisms (Sharing, Requesting, Referring, Linking) and multiple user interface functionalities. Experimental results indicate that CoPrompt significantly improves collaborative programming efficiency, reduces programmers' cognitive workload, and facilitates collaborative workflows in prompt engineering. These findings provide valuable insights for the design of future collaborative programming tools.