Computer Science Achievement and Writing Skills Predict Vibe Coding Proficiency

Paper Title

Computer Science Achievement and Writing Skills Predict Vibe Coding Proficiency

Publication Info

• Topic area: Relationship between academic skills and proficiency in LLM-driven programming workflows.
• Keywords: Vibe coding, GUI-oriented applications, LLM programming, computer science achievement, written communication skills, cognitive ability, prompt quality, cross-sectional study.

Background and Problem

• Problem / challenge: The skills that predict success in LLM-driven programming workflows, specifically vibe coding, are not well understood. This limits the ability to design effective tools and curricula for future developers.
• Significance: Understanding these predictors can guide educational priorities and tool development, especially as LLM-driven programming becomes increasingly common.
• Motivation and related work: Prior research has explored natural-language programming and the role of communication in software development but lacks direct participant-level assessments of writing and CS skills in LLM-augmented workflows. This study addresses the gap by focusing on GUI-oriented vibe coding tasks.

Solution

• Proposed approach: A cross-sectional study assessing how written communication skills, computer science (CS) achievement, and domain-general cognitive skills predict GUI-oriented vibe coding performance.
• Novelty:
  1. Empirical evidence linking written communication and CS achievement to vibe coding performance.
  2. Development of a validated assessment suite for GUI-oriented vibe coding tasks.
  3. Creation of a purpose-built vibe coding platform for controlled experimentation.
• Procedure and key techniques:
  • Participants (N=100) completed assessments of CS achievement (SCS1 subset), written communication (essay task), and cognitive ability (ICAR16).
  • They performed three vibe coding tasks (replication, feature addition, and decontextualized tasks) in a custom platform that concealed code and focused on iterative natural-language prompting.
  • Performance was scored using predefined rubrics, and statistical analyses included correlations, regressions, and mediation analysis.

Results

• Concrete findings:
  • CS achievement (r = 0.39) and writing skills (r = 0.29) significantly predict vibe coding performance.
  • CS achievement remains a significant predictor after controlling for cognitive ability, while the correlation for writing becomes non-significant.
  • CS achievement explains twice the unique variance in vibe coding performance compared to writing skills.
  • Lexical diversity and prompt quality mediate the relationship between writing skills and vibe coding outcomes.
• Advantage over baselines:
  • Demonstrates that both CS and writing skills independently contribute to vibe coding performance, with CS achievement having a stronger effect.
  • Provides evidence that prompt quality links writing skills to task success.
• Experiments / evaluation:
  • Tasks included replication, feature addition, and decontextualized GUI-oriented applications.
  • Statistical analyses included Pearson correlations, hierarchical regressions, and mediation analysis.
  • Participants’ essays and prompts were graded using expert-designed rubrics.
• Limitations and future work:
  • Focused on GUI-oriented vibe coding; results may not generalize to other domains like data analysis.
  • Limited to "no-code" vibe coding workflows; mixed-mode environments with code access were not studied.
  • Sample restricted to university students, limiting generalizability to professional developers or citizen programmers.
  • Future studies should explore causal mechanisms, broader task types, and diverse populations.

Summary

This study investigates how computer science achievement and written communication skills predict proficiency in GUI-oriented vibe coding, a workflow where code is hidden, and programming is done via natural-language prompts. Results show that both skills significantly contribute to performance, with CS achievement having a stronger impact. Writing skills influence outcomes through prompt quality, as evidenced by mediation analysis. These findings suggest that CS education and writing instruction can enhance LLM-guided programming outcomes and inform the design of tools and curricula. Future work should explore broader task types, mixed-mode workflows, and diverse populations to generalize these insights.