OSINT Clinic: Co-designing AI-Augmented Collaborative OSINT Investigations for Vulnerability Assessment

Research Background and Issues

What problems or challenges did the authors identify?

• Small businesses often become primary targets of cybercrime, accounting for 43% of total data breaches. However, they typically lack awareness of the cyber risks they face and how to mitigate them.
• Cyber vulnerability assessments require highly specialized skills and resources. Even if small businesses employ IT specialists, they often lack specific cybersecurity training.
• Traditional cybersecurity clinic models face scalability challenges. Training students from multidisciplinary backgrounds consumes significant time and resources, and comprehensive vulnerability assessments can take months to complete.

Why is this issue important?

• Small businesses have limited resources, but their cybersecurity directly impacts their business operations and data safety.
• Providing low-cost vulnerability assessment services not only enhances the cybersecurity defenses of small businesses in the community but also offers practical opportunities for future cybersecurity professionals, addressing the industry's demand for skilled talent.

Research Motivation and Related Work

• Traditional cybersecurity clinic models have contributed to improved community cybersecurity and student training opportunities, but innovation is needed to address scalability issues.
• The authors recognized the potential of using open-source intelligence (OSINT) data, as it does not require direct interaction with target systems ("zero-touch"), poses low risks, and can serve as a scalable solution for vulnerability assessments.
• However, OSINT investigations are complex and involve processing large amounts of unstructured public data. Analyzing such data is time-consuming and faces challenges such as verification and noise. The study explores how generative AI (e.g., ChatGPT) can bridge the knowledge gap in OSINT techniques.

Solution

What methods or solutions did the authors propose?

• The authors proposed the concept of the "OSINT Clinic," combining student learning with vulnerability assessment work based on open-source intelligence, focusing on addressing technical and collaborative challenges.
• They designed tools using generative AI (e.g., ChatGPT and Team-GPT) to support student training and practical work, helping students overcome major bottlenecks in OSINT analysis and improve efficiency.

What is innovative about this solution?

1. Scalable OSINT Clinic Model: Employs a "zero-touch" open-source intelligence approach, optimizing processes through generative AI.
2. Generative AI Applied to OSINT: Utilizes generative AI to support key stages such as data collection, processing, analysis, and report generation, enabling low-experience teams to deliver efficient and secure vulnerability assessments.
3. Human-AI Collaborative Design Approach: Uses a co-design methodology where students and generative AI interact continuously to create AI design improvements tailored to practical needs.

What are the implementation steps?

The authors tested the capabilities of generative AI and explored the OSINT clinic model through a three-phase co-design research process:

1. Phase 1: Identify challenges students face across the five key intelligence cycle stages (planning, data collection, processing, analysis, and dissemination) and set learning objectives.
2. Phase 2: Test how generative AI (e.g., ChatGPT) can address these challenges incrementally, such as task decomposition, template creation, and data verification.
3. Phase 3: Pilot real-world vulnerability assessment tasks with three small businesses, observing how generative AI performs in practical scenarios and evaluating the solution's value and limitations.

Research Outcomes

What specific outcomes were achieved?

• Through the OSINT Clinic model, students were able to complete cybersecurity vulnerability assessments based on public data, providing actionable security recommendations to clients.
• Client feedback indicated that the reports and recommendations generated by the clinic model were highly actionable and directly helped small businesses improve their cybersecurity.
• Generative AI (particularly ChatGPT) significantly lowered the complexity barrier for analysis, enabling efficient task decomposition, template generation, and information synthesis and interpretation.

What are the advantages compared to existing solutions?

• Scalability: The OSINT model focuses on public data, reducing time and resource costs.
• Education-Practice Integration: Students learn and apply OSINT techniques in real-world tasks, enhancing their data processing and interpretation skills.
• AI Module Optimization: Generative AI supports end-to-end collaboration, from planning to delivery, improving consistency and accuracy.

What were the experimental or evaluation results?

• Improved Student Performance: Students showed significant improvements in tasks such as template design, data analysis, and vulnerability identification.
• Client Satisfaction: All clients expressed interest in reusing the service, finding the results practical and helpful in improving their cybersecurity strategies.
• Generative AI Usability: AI effectively automated many repetitive tasks but faced limitations in privacy protection (e.g., data anonymization) and contextual understanding.

Limitations and Future Directions

• Privacy Concerns: Generative AI (e.g., ChatGPT) requires cloud-based data processing, raising privacy issues, especially when handling sensitive information.
• Collaboration Challenges: AI-generated content can vary in format and style, making it difficult to maintain consistency, and asynchronous collaboration poses challenges for progress monitoring.
• Future Optimization Directions:
  1. Explore privacy-preserving AI (e.g., locally deployed models).
  2. Deepen integration with the specific needs of small businesses.
  3. Develop more agile collaborative AI tools to enhance team efficiency in real-world task scenarios.

Conclusion

The OSINT Clinic model demonstrates how generative AI can help student teams overcome technical and collaborative obstacles, improve vulnerability assessment efficiency, and provide tangible benefits to small businesses in the community. This practice offers valuable insights into future collaborative models integrating artificial and human intelligence, while also opening up broader possibilities for educational and practical cybersecurity clinic models.