Better Together: The Interplay Between a Phishing Awareness Video and a Link-centric Phishing Support Tool

Document Title

Better Together: The Interplay Between a Phishing Awareness Video and a Link-Centric Phishing Support Tool

Document Information

• Topic Area: Research on the combination of user education and support tools against phishing attacks
• Keywords: Anti-phishing, online study, phishing awareness, video education, protection tools, user research, risk control, human-computer interaction

Research Background and Issues

• Problems and Challenges:

  • Email phishing attacks remain a significant threat in daily life and business, targeting sensitive data (e.g., passwords) or installing malware.
  • Despite progress in automated detection systems, phishing emails cannot be entirely blocked.
  • Human users still struggle to distinguish phishing emails from legitimate ones, especially when identifying URLs.

• Research Importance:

  • Enhancing users' ability to identify phishing content can significantly reduce potential security risks.
  • Comparing the effects of phishing awareness videos (e.g., NoPhish video) and support tools (e.g., TORPEDO warnings), as well as their combined use, can optimize the design of user education and support.

• Research Motivation and Existing Issues:

  • Current phishing education and support tools (e.g., NoPhish video and TORPEDO warnings) have individually demonstrated effectiveness, but their combined impact has not been thoroughly studied.
  • Exploration is needed to determine the relative advantages of tools and videos in helping users identify phishing URLs and whether they are redundant or synergistic.

Solution

• Methods and Design:

  • This study compares three approaches through an online experiment (n=409):
    1. Using the NoPhish video.
    2. Using the TORPEDO tool (providing real-time link-based warnings).
    3. Combining the NoPhish video and TORPEDO tool.
  • Investigates the independent and combined effects of proactive video education and real-time warning tools.

• Innovations:

  • First evaluation of the individual and combined effects of these methods within a unified experimental setting.
  • Provides an efficient research framework for complex phishing URL manipulation techniques (e.g., path errors, subdomain confusion).

• Experimental Steps and Key Techniques:

  1. Study Groups:
     • Multiple comparison groups (including tool use/no tool use, standalone video/tool/combined approach).
  2. Tasks and Methods:
     • Participants judge whether 16 emails are phishing or legitimate.
     • Different groups are provided with tools (TORPEDO extension) or educational videos (NoPhish).
  3. Evaluation Metrics:
     • Sensitivity and bias based on Signal Detection Theory.
     • Effectiveness analysis, including accuracy, user bias, and recognition of different phishing tactics.

Research Results

• Findings:

  1. Using the NoPhish video alone (sensitivity increased to 2.12) or the TORPEDO tool alone (sensitivity increased to 2.51) significantly improves phishing email detection rates.
  2. Combining both (sensitivity increased to 2.92) outperforms standalone approaches, demonstrating a synergistic effect.
  3. The TORPEDO tool's risk-level warnings for links effectively support users, even when facing complex URL manipulations.

• Advantages Over Existing Approaches:

  • The TORPEDO tool outperforms the NoPhish video, especially in real-time support (e.g., path checks, subdomain validation).
  • The combined approach significantly reduces "phishing misjudgments" and "false positives for legitimate addresses."

• Experimental and Evaluation Results:

  • For different phishing URL manipulation strategies (e.g., IP obfuscation, subdomain spoofing), the combined approach achieves significantly higher accuracy than traditional methods.
  • However, "minor domain modifications" remain challenging to detect (e.g., changing netflix.com to netfllx.com), highlighting areas for tool improvement.

• Limitations and Future Directions:

  1. Limitations:
     • The experimental environment is based on laboratory settings and does not fully simulate real user browsing experiences.
     • Certain interference factors (e.g., time pressure) were not adequately considered.
  2. Future Improvements:
     • Further exploration of how to improve video content and link display designs to make minor domain modifications easier to distinguish.
     • Long-term impact studies in real-world environments and adaptation analyses across different user groups.

Summary and Conclusion

• Key Findings:

  1. Phishing awareness education videos (e.g., NoPhish) and real-time support tools (e.g., TORPEDO) are effective individually, but their combination significantly enhances users' discrimination abilities.
  2. Browsers and email clients need to improve URL information display methods (e.g., adding embedded warnings and domain highlighting).
  3. Combining education and tools in practice may offer the best anti-phishing protection strategy.

• Recommendations:

  • Combining video training and tools in practical online applications can play a greater role in user education and cybersecurity.