It's a Match - Enhancing the Fit between Users and Phishing Training through Personalisation

Research Background and Problem

• Problem and Challenges: The authors identified the prevalence of a "one-size-fits-all" approach in traditional anti-phishing training, which overlooks individual differences in users' knowledge, experience, and skills, thereby limiting the effectiveness of such training.
• Significance: Phishing is one of the major cybersecurity threats, with the number of related emails continuously increasing. Current technical defense measures are insufficient to replace the human ability to effectively identify phishing attempts, making it crucial to enhance users' phishing detection capabilities.
• Research Motivation: The potential of personalized training has been demonstrated in the education field, but there is a lack of systematic empirical research in phishing or broader cybersecurity training. Additionally, resource and budget constraints, as well as data privacy concerns, hinder the implementation of personalized training. This study aims to validate the effectiveness of a simple modular personalized approach, laying the groundwork for future advanced personalization research.

Solution

• Method: The authors proposed a personalized training scheme based on users' phishing detection abilities and evaluated it in an online study. A total of 342 participants were assigned to receive either personalized training, randomly assigned training, or basic training through educational videos only.
• Innovations:
  1. Personalized Assessment Model: A comprehensive scoring method was used to categorize participants into low, medium, and high ability groups based on their phishing detection skills, knowledge levels, and security attitudes.
  2. Modular Design: Training content was tailored to the three ability levels—educational videos, interactive quizzes, and warning banners.
  3. Data Privacy-Friendly Approach: The personalization method did not rely on sensitive data (e.g., age or other potentially discriminatory information).
• Implementation Steps:
  1. Collect users' ability data through preliminary questionnaires and tests, and assign them to corresponding training groups using the scoring model.
  2. Assign training content based on the needs of each ability group. The low-ability group received the full training package, the medium-ability group received quizzes and banners only, and the high-ability group received banners only.
  3. Conduct post-training tests to evaluate the effectiveness of the training.

Research Findings

• Specific Findings:
  1. Significant Training Effectiveness: Phishing detection abilities improved significantly across all ability groups, with the low-ability group showing the most notable improvement.
  2. Advantages of Modularity: Interactive quizzes and warning banners played a key role in reinforcing knowledge and improving vigilance, while educational videos enhanced users' foundational understanding.
  3. Preference for Personalization: Users generally preferred personalized allocation, especially when the training content matched their ability levels.
  4. Efficiency Improvement: Personalized training reduced the time spent on unnecessary content, making the learning process more targeted.
• Comparison with Existing Methods:
  • Compared to the random assignment group and the video-only group, the personalized allocation group showed greater improvements in phishing detection ability, self-assessed knowledge, and security attitudes.
  • While educational videos were effective, their impact was less pronounced when used alone compared to when combined with interactive components.
• Experimental and Evaluation Results:
  1. The personalized group achieved an average improvement of 19% in phishing detection ability (measured by test accuracy).
  2. Random group assignments that did not match users' abilities led to less consistent improvement.
  3. Training that matched users' ability levels resulted in more significant improvements in both overall ability and subjective evaluations.
• Limitations and Future Directions:
  1. Sample Representativeness: The study primarily focused on English-speaking users from Western regions, lacking validation across other cultural and linguistic contexts.
  2. Lack of Long-Term Effects: The study did not examine the retention of training effects over several months, which should be addressed in future follow-up studies.
  3. Needs of Advanced Users: While the training was highly effective for lower-ability groups, further research is needed to enhance training for high-ability users, such as incorporating in-depth technical details or more complex scenario simulations.
  4. Potential for Dynamic Personalization: Future studies could leverage AI tools to further optimize personalization and adjust training content based on users' professional roles or industry characteristics.

Through this study, the authors successfully enhanced the effectiveness of phishing training using a simple modular personalized approach, providing practical and theoretical support for the design of advanced personalized training in the future.