RepliCueAuth: Validating the Use of a Lab-Based Virtual Reality Setup for Evaluating Authentication Systems

Document Title

RepliCueAuth: Validating the Use of a Lab-Based Virtual Reality Setup for Evaluating Authentication Systems

Document Information

• Subject Area: Virtual Reality (VR), Usable Security, Authentication Systems
• Keywords: Virtual Reality, Research Methods, Usable Security, Authentication, Observation Resistance, User Experience, Input Methods, Lab Experiments, User Studies, Technology Evaluation

Research Background and Issues

• Problems and Challenges:

  • Evaluating novel authentication systems is often costly and time-consuming, especially for systems involving non-commercialized devices (e.g., private near-eye displays, eye trackers).
  • Conducting related field evaluations with physical prototypes (e.g., "in-the-wild" studies) is complex and expensive.
  • Investigating whether evaluations conducted in a virtual reality (VR) environment can provide comparable data to real-world authentication scenarios.

• Significance:

  • If VR can enable rapid, cost-effective iterative studies of authentication prototypes while producing reliable data, it would significantly enhance research efficiency and scalability in the field of usable security.
  • VR studies allow for the evaluation of user interaction, performance, and resistance to attacks without requiring physical devices or being affected by real-world distractions.

• Research Motivation:

  • To explore the feasibility and limitations of using VR as a testing platform for evaluating real-world authentication systems and to compare the correlation between VR-based experimental results and real-world evaluation outcomes.

Solution

• Methods and Implementation:

  • A proof-of-concept study was proposed to use VR as a platform for evaluating the usability and security of authentication systems.
  • A real-world authentication system experiment, CueAuth, was replicated in a VR environment, and two studies were conducted:
    1. Lab-Based VR Usability Study (N=20): Investigated user performance in authentication using touch, mid-air gestures, and eye gaze.
    2. Online Survey-Based Security Study (N=22, observing recorded videos of virtual avatars): Examined the system’s resistance to shoulder-surfing attacks under two threat models.
  • The data obtained was compared with the original real-world CueAuth experiment results.

• Innovations:

  • First comprehensive attempt to validate the alignment of VR experiments with real-world usability and security evaluations.
  • Utilized virtual avatars in security experiments, reducing physical requirements for equipment and venues, thereby enhancing experimental flexibility.
  • Provided specific experimental strategies and standardized design and execution frameworks, improving the reproducibility of similar future studies.

• Implementation Steps and Key Technologies:

  • Built the VR system using Unity3D, incorporating Leap Motion and Tobii XR for hand and eye tracking.
  • Provided haptic feedback by mapping a real touch surface to a virtual display to simulate a VR touchscreen experience.
  • Employed specially designed questionnaires and existing psychological measurement tools (e.g., NASA-TLX) to assess cognitive load and user experience.
  • Designed two threat models for the security experiment: "single attack" and "repeated video attack."

Research Outcomes

• Key Findings:

  • User performance in the VR environment (e.g., input accuracy) showed a high correlation with real-world experiments.
    • No significant differences in input accuracy among the three input methods.
    • Eye gaze authentication using head-mounted VR devices demonstrated higher accuracy and user preference compared to fixed eye trackers in the original study.
    • Touch input performance was affected by the representation of virtual hands, leading to significantly longer input times.
  • Security experiments using virtual avatars to evaluate observation resistance yielded attack success rates closely matching real-world results.

• Advantages Compared to Existing Solutions:

  • Demonstrated significant potential for conducting large-scale experiments in VR without additional physical costs.
  • Generated user experience and performance data comparable to real-world environments while avoiding on-site distractions and observer effects.

• Experimental or Evaluation Results:

  • Usability Experiment Results:
    • Impact of input methods on user experience: Touch > Mid-air Gestures > Eye Gaze (original study), but eye gaze was significantly favored in the VR environment due to hardware advantages.
    • Cognitive load and subjective evaluations: The three methods were generally consistent across NASA-TLX dimensions.
  • Security Experiment Results:
    • Eye gaze was the most secure input method overall, with consistent shoulder-surfing resistance results across both study setups.
    • Repeated video attacks significantly increased the vulnerability of mid-air gestures and touch input.

• Limitations and Future Directions:

  • The study only explored the VR mapping of one specific authentication system; broader application to diverse authentication technologies is needed.
  • Future research should focus on enhancing the ecological validity of VR experiments (e.g., simulating real-world external distractions such as ambient lighting and crowd pressure).
  • Investigate the potential of remote VR participant experiments to maximize the effectiveness of distributed device collaboration.