"The AI tool can’t make it any worse." Investigating Developers’ Security Behavior with AI Assistants in a Password Storage Study

Paper Title

"The AI tool can’t make it any worse." Investigating Developers’ Security Behavior with AI Assistants in a Password Storage Study

Publication Info

• Topic area: Developer security practices and AI-assisted programming.
• Keywords: AI assistants, password storage, secure coding, developer behavior, security instructions, ChatGPT, GitHub Copilot, usability, cryptography, software development.

Background and Problem

• Problem / challenge: Developers often neglect secure password storage practices without explicit instructions, and the impact of AI assistants on these practices remains unclear.
• Significance: Secure password storage is critical to protecting user data from offline attacks, yet insecure implementations can expose millions of users to risks.
• Motivation and related work: Previous studies demonstrated the effectiveness of explicit security instructions but were conducted before AI assistants became widespread. Recent research highlights both the potential and risks of AI-generated code, necessitating a reevaluation of established security interventions in AI-assisted workflows.

Solution

• Proposed approach: Two studies—Study A (qualitative lab study with 21 students) and Study B (quantitative online study with 80 freelance developers)—investigating the interplay between AI assistance and explicit security instructions in password storage tasks.
• Novelty:
  1. Demonstrates that explicit security instructions remain effective in AI-assisted workflows.
  2. Shows how AI assistants influence developers’ security behavior and decision-making.
  3. Highlights the nuanced relationship between trust in AI-generated code and practical reliance on it.
• Procedure and key techniques:
  • Participants were assigned to four instruction conditions: non-instructed, AI-instructed, security-instructed, and security-plus-AI-instructed.
  • Data collection included screen recordings, surveys, interviews, and code analysis using a 0–7 security scoring scale.
  • Hypotheses tested the effects of instructions and AI usage on security scores.

Results

• Concrete findings:
  • Security instructions significantly improved security scores in both studies.
  • Combining security and AI instructions yielded the most secure outcomes (e.g., mean security score of 5.1/7 in Study A).
  • AI-assisted solutions achieved higher security scores compared to non-AI solutions, especially when paired with explicit instructions.
• Advantage over baselines:
  • Compared to pre-AI studies, AI-assisted workflows improved security scores by 0.5 points on average, with one participant achieving the academic standard of 7 points.
  • Post-task security instructions increased scores notably for groups without pre-task instructions.
• Experiments / evaluation:
  • Study A: Lab-based setup with 21 students implementing password storage in Java using Eclipse IDE and PostgreSQL.
  • Study B: Remote study with 80 freelance developers using virtual machines and diverse tools.
  • Metrics included functionality, security scores, and participant perceptions of AI-generated code.
• Limitations and future work:
  • Observational bias in lab settings and fatigue from long task durations.
  • Usability challenges with third-party libraries influenced participants’ security decisions.
  • Future research should explore adaptive AI systems, real-time detection of misconceptions, and replication across other security-critical tasks.

Summary

This paper investigates how AI assistants influence developers’ security practices in password storage tasks. Two studies revealed that explicit security instructions remain effective in AI-assisted workflows and that combining security and AI instructions yields the most secure outcomes. AI-assisted solutions improved security scores compared to pre-AI studies, but participants often prioritized convenience over security and expressed skepticism toward AI-generated security suggestions. The findings highlight the importance of explicit instructions, validation guidance, and usability improvements in AI tools to ensure secure coding practices.