Precision Email Simulator for Research on Safety-Critical Phishing Behaviour

Research Background and Issues

• Identified Problems or Challenges:

  • The authors highlight that despite extensive research on phishing susceptibility, the factors influencing users' micro-behaviors during interactions with phishing emails remain insufficiently understood.
  • Most current studies overlook the potential of simulation experiments in phishing behavior research, which is an effective method combining laboratory control with real-world scenarios.

• Significance:

  • As email becomes a critical tool for personal and professional communication, its associated risks are also increasing. Phishing can lead to severe consequences, such as financial losses and sensitive information leaks.
  • The impact of phishing emails often occurs during brief moments of user interaction with the email, making it crucial to understand users' real-time decision-making behaviors for designing targeted training and interventions.

• Research Motivation and Related Work:

  • Inspired by existing studies (e.g., experimental email management simulation research by Akbar et al. and Kerr et al.), the authors believe that a simulated email client offers an ideal method to study the decision-making dynamics of users interacting with phishing emails.
  • Most current phishing research relies on surveys or simulated phishing emails, which lack in-depth observation of user interactions and causal factors.
  • Experimental simulation tools can overcome these limitations, and related literature indicates that such methods are still underdeveloped.

Proposed Solution

• Proposed Solution:

  • The authors developed an open-source tool called "Precision Email Simulator" to support phishing behavior research and enable high-precision user data analysis, such as email reading time and eye-tracking.
  • The tool simulates a realistic email interface, allowing researchers to control email content, task complexity, and real-time interaction conditions.

• Innovative Features:

  • The tool supports comprehensive user behavior recording, including mouse clicks, keyboard inputs, eye-tracking data, and user actions on emails (e.g., commenting, forwarding, marking).
  • It provides a controlled laboratory environment while maintaining the realism of email interactions, enabling causal inference, such as investigating how user workload affects phishing email detection.
  • It integrates third-party research tools (e.g., iMotions) to expand data collection dimensions.

• Implementation Steps and Key Techniques:

  • Study Design: Create detailed scenarios for participants, including role-playing and task objectives (primary and secondary tasks).
  • Email Content Design: Develop realistic email content to simulate an authentic inbox environment, including a mix of regular emails, advertisements, and phishing emails.
  • Tool Configuration and User Data Collection: Adjust the tool interface to fit different research tasks, collect micro-interaction data, and integrate eye-tracking and biosensor data.
  • Experiment Management and Setup: The tool supports flexible experimental designs, such as comparing high workload and low workload conditions.

Research Outcomes

• Specific Findings:

  • The authors shared results from two experimental studies exploring the impact of workload on phishing email detection ability.
    • Experiment 1 found that under high workload, participants spent less time processing emails and paid more attention to task-relevant phishing emails.
    • Experiment 2 further validated how workload affects users' attention to phishing cues, such as sender addresses, and their interactions with phishing links.

• Advantages:

  • Compared to existing phishing research methods, this tool not only collects precise data on users' decision-making processes but also significantly extends research by simulating realistic scenarios.
  • It allows the study of user behavior under standardized and highly controlled conditions while enabling causal inference on micro-interactions.
  • It improves upon traditional methods, such as surveys or simple observations, which fail to capture dynamic data.

• Experimental or Evaluation Results:

  • The results of the two experiments confirmed that phishing susceptibility is significantly influenced by workload and task relevance, suggesting that researchers should consider environmental variables when optimizing phishing interventions.
  • The experiments also showed that participants rated the simulation tool highly for scenario realism and task richness.

• Limitations and Future Directions:

  • Limitations: Participant behavior in simulation experiments may differ slightly from real-world scenarios, as the consequences of tasks are relatively minor, leading to less engagement.
  • Future Directions:
    • Add a questionnaire module to the tool, including personality traits and workload assessments.
    • Enhance the realism of the simulation environment, such as interactive email attachments and precise eye-tracking regions.
    • Expand the experimental logic of phishing emails, applying the tool to long-term studies and behavioral training design.

In summary, the "Precision Email Simulator" offers a highly promising research platform that supports extensive phishing behavior studies and advances the understanding of user behavior in security-critical tasks.