“Helping Me Versus Doing It for Me”: Designing for Agency in LLM-Infused Writing Tools for Science Journalism

Paper Title

“Helping Me Versus Doing It for Me”: Designing for Agency in LLM-Infused Writing Tools for Science Journalism

Publication Info

• Topic area: The impact of LLM-infused tools on journalists' agency in editorial decision-making.
• Keywords: Science journalism, generative AI, agency, value-sensitive design, pitch writing, autonomy, professional values, LLM tools, creativity, editorial workflows.

Background and Problem

• Problem / challenge: The adoption of generative AI in journalism risks eroding journalists' agency by reshaping editorial decision-making and execution, potentially undermining professional values like autonomy, critical thinking, and skill development.
• Significance: Journalists play a critical role in sustaining informed public discourse, and preserving their agency ensures alignment with values such as transparency, accuracy, and public welfare.
• Motivation and related work: Prior research highlights how generative AI tools can shift control from users to systems, influencing decisions and reducing critical engagement. However, there is limited understanding of how these dynamics intersect with the professional values of journalists, particularly in the context of pitch writing.

Solution

• Proposed approach: The study investigates how journalists negotiate agency with LLM-infused writing tools by presenting four hypothetical tool designs and analyzing their responses.
• Novelty:
  1. Identification of nine professional values guiding journalists’ willingness to cede control to AI tools.
  2. Analysis of conditions under which AI tools threaten journalistic agency, such as constraining exploration or bypassing reflective work.
  3. Design implications for creating agency-preserving tools that balance efficiency with decision-making control.
  4. Characterization of the interplay between causal and identity-based agency in sustaining professional identity over time.
• Procedure and key techniques:
  • Semi-structured interviews with 20 freelance science journalists.
  • Presentation of four speculative LLM-infused tool concepts varying in task initiation, scope of AI assistance, and configurability.
  • Reflexive thematic analysis to identify value tensions and boundaries of control.

Results

• Concrete findings:
  • Journalists value autonomy, fulfillment, skill development, and distinctiveness in their craft, alongside contextual values like efficiency, ethical standards, and editorial alignment.
  • AI assistance for information gathering and feedback generation was widely accepted, while core idea and text generation were resisted due to concerns about autonomy, voice, and reflective labor.
  • Economic pressures and situational constraints (e.g., tight deadlines) occasionally led to acceptance of AI-generated drafts, but these were compromises rather than value-driven choices.
• Advantage over baselines:
  • The study provides a nuanced understanding of how journalists’ professional values shape their interaction with AI tools, offering insights beyond general creative writing contexts.
  • It highlights the importance of designing tools that connect users to resources and feedback rather than generating content directly.
• Experiments / evaluation:
  • Four tool concepts were evaluated: Pitch Assist (open-ended chat), Pitch Refine (scaffolded subtasks), Pitch Critic (feedback during writing), and Pitch Suggest (automatic drafting).
  • Participants ranked Pitch Critic highest overall, followed by Pitch Suggest for its information-gathering capabilities.
  • Evaluation metrics included participants’ rankings, qualitative feedback on control boundaries, and alignment with professional values.
• Limitations and future work:
  • The study focuses on freelance science journalists, which may amplify concerns about economic precarity and autonomy.
  • Longitudinal studies are needed to assess the long-term impacts of AI use on skill development and professional identity.
  • Future work should explore configurability and systemic factors influencing AI adoption in journalism and other creative professions.

Summary

This study explores how LLM-infused writing tools affect journalists’ agency, focusing on pitch writing in science journalism. Through interviews with 20 journalists, it identifies nine professional values that shape their boundaries of control over AI assistance. While journalists accept AI for information gathering and feedback, they resist its use in core ideation and text generation due to concerns about autonomy, voice, and reflective labor. The findings emphasize the interplay between causal and identity-based agency, highlighting the need for tools that support efficiency without undermining professional values. The study offers design implications for preserving agency in journalism and other creative domains, while calling for longitudinal research to address systemic and temporal challenges.