Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight

Paper Title

Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight

Publication Info

• Topic area: Interaction between GUI agents and manipulative interface designs (dark patterns).
• Keywords: Dark patterns, GUI agents, human oversight, LLMs, task automation, user autonomy, deception, transparency, cognitive load, agent vulnerability.

Background and Problem

• Problem / challenge: GUI agents, powered by large language models (LLMs), are increasingly deployed to automate tasks on graphical user interfaces (GUIs). However, their susceptibility to manipulative interface designs (dark patterns) remains poorly understood. These agents often prioritize task completion over user safety and privacy, potentially misaligning their actions with user interests.
• Significance: Understanding and mitigating GUI agents' vulnerabilities to dark patterns is critical for ensuring user safety, trust, and privacy in automated systems, especially as these agents are deployed in high-stakes domains like finance and healthcare.
• Motivation and related work: Prior research has focused on human susceptibility to dark patterns and on LLM vulnerabilities to adversarial prompts. However, the unique challenges posed by dark patterns to GUI agents, including their cognitive and perceptual limitations, have not been systematically studied. This paper addresses this gap by evaluating agent and human responses to dark patterns and exploring the role of human oversight.

Solution

• Proposed approach: A two-phase empirical study to evaluate GUI agents' and humans' responses to 16 types of dark patterns, with a focus on agent vulnerabilities, human-agent differences, and the impact of human oversight.
• Novelty:
  1. Systematic evaluation of GUI agents' susceptibility to dark patterns across diverse scenarios and agent types.
  2. Direct comparison of human and agent vulnerabilities to dark patterns, highlighting distinct failure modes.
  3. Analysis of human oversight's effectiveness in mitigating dark patterns and the associated cognitive and attentional costs.
• Procedure and key techniques:
  • Phase 1: Evaluated six GUI agents (four adapted LLM-based and two end-to-end agents) on 16 dark patterns, measuring task completion, awareness, and avoidance rates.
  • Phase 2: Conducted a within-subjects experiment with 22 human participants to compare their responses to dark patterns in two conditions: working independently and supervising a GUI agent (Operator).
  • Data collection included reasoning traces, task outcomes, and participant interviews to analyze awareness, avoidance, and oversight behaviors.

Results

• Concrete findings:
  • GUI agents often avoided dark patterns incidentally without recognizing them, with awareness rarely leading to protective actions.
  • Humans and agents failed on similar dark patterns (e.g., bad defaults, hidden information) but for different reasons: humans relied on heuristics, while agents prioritized task completion.
  • Human oversight improved avoidance rates but introduced costs such as attentional tunneling, cognitive load, and reduced awareness of manipulative designs.
• Advantage over baselines:
  • End-to-end agents outperformed adapted LLM-based agents in using termination as a safety mechanism (e.g., pausing for user confirmation).
  • Human-agent teams outperformed humans or agents alone in avoiding certain dark patterns but at the expense of increased cognitive demands.
• Experiments / evaluation:
  • Phase 1: Tested six agents on 16 dark patterns across e-commerce, social media, and video streaming domains.
  • Phase 2: Reused the same tasks and dark patterns, comparing human-only and human-agent supervision conditions with a Latin Square design to counterbalance task and condition orders.
• Limitations and future work:
  • Rapidly evolving GUI agent architectures may shift vulnerabilities over time.
  • Controlled experiments with isolated dark patterns may not fully capture real-world complexities where multiple patterns co-occur.
  • Small sample size (22 participants) limits generalizability; future studies should include larger and more diverse populations.
  • Retrospective reasoning traces and interview-based data may not fully reflect real-time decision-making processes.

Summary

This study investigates the susceptibility of GUI agents and humans to manipulative dark patterns and evaluates the role of human oversight. It finds that GUI agents often fail due to procedural blind spots, while humans rely on heuristics and habitual compliance. Human oversight improves avoidance rates but introduces new vulnerabilities, such as attentional tunneling and cognitive overload. The findings highlight the need for improved agent transparency, adaptive autonomy, and informed oversight mechanisms to balance efficiency with user safety and autonomy. These insights are critical for the safe deployment of GUI agents in high-stakes domains and for developing regulatory frameworks to address agent-mediated interactions.