Generative AI Uses and Risks for Knowledge Workers in a Science Organization

Research Background and Issues

• What problems or challenges did the authors identify?
  The authors identified that although Generative AI holds significant potential to enhance the efficiency of scientific organizations by supporting knowledge workers, its practical applications and potential risks within such organizations remain unclear. These uncertainties pose challenges for scientific organizations in fully leveraging the capabilities of Generative AI.

• Why is this issue important?
  Scientific organizations are tasked with advancing scientific progress, such as supporting drug development and climate solutions. If Generative AI can accelerate research processes in these scientific fields, it could have a profound positive impact on society. Additionally, Generative AI may transform the way scientific research and daily operations are conducted, which is critical for the efficiency and safety of knowledge workers.

• Research Motivation and Related Work
  The research motivation stems from the underexplored areas of Generative AI in scientific work settings and heightened concerns about its potential risks. Existing literature has focused on the advantages of Generative AI in areas like coding and automating scientific tasks, but there is limited attention to its overall impact on scientific work and the interaction between science and operations. Furthermore, there is a lack of research on privacy and security risks, especially in sensitive data environments like scientific organizations.

Solutions

• What methods or solutions did the authors propose?
  The authors analyzed the current and future applications and risks of Generative AI through surveys and case studies conducted in U.S. national laboratories. They designed a study involving a questionnaire survey (N=66), interviews (N=22), and usage data collection of an internal Generative AI tool—Argo—to address the identified issues.

• What are the innovative aspects of this solution?
  The authors' research makes groundbreaking contributions in the following areas:

  1. Collected organizational-level data on the deployment of an internal Generative AI tool, providing insights into real-world applications.
  2. Categorized user scenarios of Generative AI into two interaction modes: "Copilot" and "Workflow Agent."
  3. Investigated the unique perspectives of scientific and operational staff on the risks of Generative AI.
  4. Offered specific recommendations for designing Generative AI tools for scientific and knowledge-based organizations.

• What are the implementation steps and key technologies used?

  1. Questionnaire Survey: Assessed employees' familiarity with Generative AI, usage patterns, and specific concerns (e.g., privacy and security).
  2. Interviews: Conducted in-depth semi-structured interviews to define and detail application scenarios and risks of Generative AI.
  3. Argo Tool Usage Data Analysis: Collected and analyzed metadata on Argo usage during the study period to identify early user behaviors and trends.
  4. Based on these data, the authors proposed design recommendations and future research directions for Generative AI applications.

Research Outcomes

• What specific outcomes were achieved?

  1. Usage Data: Although less than 10% of laboratory employees used Argo monthly during the study period, the number of users showed an upward trend. The practical applications of Generative AI expanded in both scientific and operational tasks.
  2. Interaction Mode Classification: Current applications were categorized as "Copilot," focusing on real-time interaction with users to complete specific tasks, while future applications were envisioned as "Workflow Agents" capable of handling more complex automated tasks.
  3. User Cases: The authors provided concrete cases, including automating experiments and data analysis in scientific teams and task management and automated project planning in operational teams.
  4. Risks: Identified risks include the unreliability of Generative AI, over-reliance, privacy and data security concerns, academic publishing issues, and potential impacts on job roles.

• What advantages does it have compared to existing solutions?
  This study addresses the research gap regarding the application of Generative AI in specific work scenarios and operational environments within scientific organizations. It also directly compares and analyzes two types of employees (scientific and operational), revealing shortcomings in existing Generative AI tools (e.g., ChatGPT) and highlighting the advantages of internally customized tools.

• What were the experimental or evaluation results?

  • "Argo," as an internal Generative AI tool, demonstrated a growing usage trend and was well-received by employees, indicating that a secure, domain-specific tool may be more suitable for scientific institutions than external tools.

• Limitations and Future Directions
  Limitations:

  • The study was limited to a single scientific organization, and the use of external Generative AI tools was not comprehensively analyzed.
  • Respondent representation may be biased, as the research sample primarily consisted of early adopters who showed a positive interest in Generative AI.
  • Gender and racial diversity were low; future studies should include a more diverse audience.

  Future Directions:

  • Design more robust Generative AI tools tailored to scientific organizations, particularly those capable of managing workflows involving complex scientific instruments.
  • Develop clear academic publishing guidelines to address issues such as citation and content reliability in Generative AI applications.
  • Enhance transparency in AI recruitment and skill selection within the industry and study its long-term societal impacts.
  • Create more generalized template-based tools and workflow agents to meet the needs of a broader range of organizations.

Through this study, the authors highlighted the potential of Generative AI in scientific organizations while identifying necessary steps to optimize reliability and manage risks. This has profound implications for the application of Generative AI in scientific discovery and knowledge work.