"Are we writing an advice column for Spock here?" Understanding Stereotypes in AI Advice for Autistic Users

Paper Title

'Are we writing an advice column for Spock here?' Understanding Stereotypes in AI Advice for Autistic Users

Publication Info

• Topic area: Stereotypes in AI-generated advice for autistic users.
• Keywords: Autism, stereotypes, LLMs, personalization, bias, advice, disclosure, HCI, neurodiversity, AI ethics.

Background and Problem

• Problem / challenge: Limited understanding of how autism disclosure affects advice provided by LLMs, and whether such advice is personalized or reinforces harmful stereotypes.
• Significance: Autistic individuals often use LLMs for advice, expecting nonjudgmental and tailored responses. However, biases in AI systems may perpetuate stereotypes, impacting the quality and inclusivity of AI-mediated support.
• Motivation and related work: Prior studies show that stereotypes about autism are encoded in LLMs and surface in generated outputs. While autistic users value AI advice for its objectivity, concerns about misrepresentation and stigma persist. This study aims to systematically examine how autism disclosure shapes LLM advice and its implications for users.

Solution

• Proposed approach: A mixed-methods study combining a large-scale LLM audit experiment and qualitative interviews with autistic participants to analyze how autism disclosure influences AI-generated advice.
• Novelty:
  1. First large-scale audit of autism stereotypes in LLM advice, analyzing 345,000 decisions across six models.
  2. Qualitative insights from autistic users on how LLM advice can feel supportive or harmful, informing design principles for personalization.
  3. Development of a six-step pipeline to operationalize 12 autism stereotypes into decision-making scenarios, enabling systematic quantification of stereotype-linked differences in LLM responses.
• Procedure and key techniques:
  1. Identify 12 autism stereotypes from prior literature.
  2. Construct and validate interpersonal decision-making scenarios reflecting these stereotypes.
  3. Generate 345,000 responses from six LLMs (Gemini-2.0-flash, GPT-4o-mini, Claude-3.5-Haiku, Llama-4-Scout, Qwen-3-235B, DeepSeek-V3) under autism disclosure (AT) and non-disclosure (NA) conditions.
  4. Measure stereotype (ST–AST) and disclosure (AT–NA) gaps in model recommendations.
  5. Conduct interviews with 11 autistic participants to contextualize quantitative findings with lived experiences.

Results

• Concrete findings:
  • Autism disclosure significantly shifted advice in 36 of 72 stereotype–model pairs, often steering toward conservative, risk-averse recommendations.
  • Four stereotypes (introverted, obsessive, aromantic, dangerous) showed the strongest disclosure effects, with models disproportionately recommending avoidance of social events, confrontations, new experiences, and romantic relationships.
  • The largest AT–NA gap was observed in social invitation scenarios, where disclosure made models 2.4 times more likely to recommend declining.
• Advantage over baselines:
  • The study goes beyond prior work by directly linking disclosure effects to stereotype-driven differences and incorporating autistic user perspectives.
• Experiments / evaluation:
  • Quantitative: Chi-squared tests and Pearson’s r correlations measured the significance and alignment of stereotype and disclosure effects.
  • Qualitative: Interviews with 11 autistic adults (ages 22–65) explored perceptions of LLM advice, stereotypes, and personalization preferences.
• Limitations and future work:
  • Synthetic scenarios and binary decision prompts may not fully capture real-world contexts.
  • Limited exploration of nuanced autism disclosures (e.g., specific traits instead of identity labels).
  • Future work should include real prompts from autistic users and analyze how LLMs adapt the style and tone of responses post-disclosure.

Summary

This study investigates how autism disclosure influences LLM-generated advice, revealing that disclosure often leads to risk-averse recommendations aligned with stereotypes of autistic individuals as introverted, obsessive, aromantic, or dangerous. While some users found this protective, others criticized it as infantilizing. The research combines large-scale quantitative audits with qualitative insights from autistic participants, highlighting the complexity of balancing personalization and bias in AI systems. The findings underscore the need for design features that give users greater control and transparency over how identity disclosures shape AI responses, paving the way for more inclusive and adaptive AI-mediated support.