How Scientists Use Large Language Models to Program

Research Background and Issues

• Problem Identification: Scientific research increasingly relies on programming, yet many scientists lack formal programming training, leading to inconsistent code quality control, particularly in testing practices common in software development. This phenomenon may result in unforeseen programming errors, jeopardizing the credibility of scientific conclusions. Meanwhile, the use of Code LLMs (code generation tools based on large language models) in program development is gradually expanding, but their reliability and scientific impact have yet to be fully assessed.
• Significance: Scientific research depends on high-quality code, and errors can lead to biases in data analysis and results. For instance, this study cites several research papers retracted due to code errors. As Code LLM usage becomes more prevalent, uncertainties related to generated code may affect experimental design parameters and even the validity of scientific analyses.
• Motivation and Related Work: While Code LLMs have garnered attention for improving programming efficiency and assisting less experienced programmers, they also pose challenges regarding the verification of generated code. Existing studies indicate that both students and professional programmers face difficulties in validating generated code, which requires further investigation and supplementation, especially in the context of scientific programming.

Solution

• Research Methods:
  • Conducting a survey of 199 scientific programmers to explore their behavioral patterns and tool preferences when using Code LLMs.
  • Performing in-depth interviews with 14 scientific researchers to gain insights into their motivations for using Code LLMs, verification strategies, and potential issues with code generation.
  • Collecting interaction logs between users and Code LLMs to analyze behaviors and problems in real-world usage scenarios.
• Research Innovation: This study combines surveys and interviews to systematically investigate the current use of Code LLMs and verification methods from the perspective of scientific programming, highlighting significant differences in behavior and outcomes between interfaces like ChatGPT and GitHub Copilot.
• Implementation Details:
  1. User Survey: Characterizing how scientists across different research fields use Code LLM tools, with a focus on the relationship between tool choice (Chat vs. Copilot), programming language habits, and usage frequency.
  2. Interview Analysis: Collecting use cases to analyze scenarios where Code LLMs are used as information retrieval tools and documenting strategies for verifying code correctness.
  3. Log Analysis: Reviewing user interactions with generated code to assess potential issues and correlate them with user behaviors.

Research Outcomes

• Key Findings:
  • Scientists predominantly use browser-based interactive tools like ChatGPT rather than IDE-integrated tools like Copilot. This trend is particularly evident in life sciences, whereas Copilot usage is higher in computer science and engineering fields.
  • Scientists primarily use Code LLMs for information retrieval, such as understanding unfamiliar libraries and language documentation, often replacing Google searches and official documentation reviews.
  • Code verification methods mainly rely on running the code to check results ("visual inspection") or manually reading line by line, but these methods are inefficient and prone to overlooking errors.
  • User logs reveal that some logical errors in Code LLM-generated code go unnoticed by users, such as incorrect parameter settings or subtle modifications to critical code segments.
• Comparison with Existing Solutions:
  • This study emphasizes the inadequacies of Code LLM verification strategies in scientific programming, uncovering misuse cases with potentially greater impacts that have not been sufficiently addressed in the broader programmer community.
  • It proposes that the "scientist user group" should be considered independently, as their coding goals are primarily research-driven rather than conventional commercial or engineering software development.
• Experimental Evaluation:
  • Data shows that multi-language programming is common in scientific fields, increasing scientists' reliance on Code LLMs, especially in cross-disciplinary collaboration scenarios.
  • The lack of a robust testing and verification culture has led researchers to transfer trust to generation tools, but this reliance, which is not entirely rational, may pose significant scientific risks.
• Limitations and Future Directions:
  • Limitations:
    • The sample is concentrated in a top-tier university in the United States, which may not fully represent the global scientific community.
    • Data is based on self-reports, which may introduce some bias.
    • The study only investigates specific tools (e.g., ChatGPT and GitHub Copilot) and does not cover all available Code LLMs.
  • Future Directions:
    • Optimizing Code LLM interactions within IDEs to reduce cognitive load during complex code output verification.
    • Designing specialized tools tailored to scientific programming needs, such as more transparent code generation feedback and automated parameter sensitivity analysis.
    • Exploring ways to reduce scientists' over-reliance on generated code through training and tool design, while promoting the establishment of a code testing culture.