LLMs Homogenize Values in Constructive Arguments on Value-Laden Topics

Paper Title

LLMs Homogenize Values in Constructive Arguments on Value-Laden Topics

Publication Info

• Topic area: The impact of large language models (LLMs) on value framing in constructive discourse on sensitive topics.
• Keywords: LLMs, value homogenization, constructive discourse, prosocial values, conservative values, cultural differences, homophobia, Islamophobia, GPT-4, ethical AI.

Background and Problem

• Problem / challenge: Limited understanding of how LLMs handle and reshape underlying values when rewriting arguments on sensitive, value-laden topics. Prior research has focused on linguistic constructiveness but not on value alignment.
• Significance: Understanding how LLMs influence value framing is critical as they are increasingly used to mediate online discourse, potentially affecting inclusivity, fairness, and the representation of diverse viewpoints.
• Motivation and related work: While LLMs have been shown to improve linguistic constructiveness, they often misalign with human values, favoring prosocial and progressive orientations over conservative ones. This study addresses the gap by systematically analyzing value shifts in LLM-rewritten comments on divisive topics.

Solution

• Proposed approach: A three-phase study to investigate how LLMs (GPT-4) rewrite comments on value-laden topics, focusing on value framing and perceived alignment with human values.
• Novelty:
  1. Systematic analysis of value shifts in LLM-rewritten comments using Schwartz’s theory of basic human values.
  2. Examination of how LLMs alter stances and value framing, often steering discourse toward prosocial directions.
  3. Cross-cultural evaluation of perceived alignment between human and LLM-rewritten comments.
• Procedure and key techniques:
  • Phase 1: Collected 234 comments from Indian and American participants on homophobic and Islamophobic threads, analyzed for value framing.
  • Phase 2: Used GPT-4 to rewrite these comments constructively, comparing linguistic and value features between human-written and LLM-rewritten versions.
  • Phase 3: Conducted a forced-choice experiment where participants evaluated which comments (human or LLM-rewritten) better aligned with their values.

Results

• Concrete findings:
  • LLMs systematically downplayed Conservative values (e.g., Tradition, Security, Conformity) and amplified prosocial values (e.g., Universalism, Benevolence).
  • LLM rewrites often shifted stances, making comments opposing same-sex marriage or Islam more neutral or supportive.
  • LLM-rewritten comments exhibited greater linguistic constructiveness (e.g., readability, politeness, reasoning markers) than human-written ones.
• Advantage over baselines:
  • LLM-rewritten comments were perceived as more aligned with values by participants supporting same-sex marriage or Islam (59.9%-61.1% overall preference).
  • However, participants opposing these topics preferred human-written comments for better alignment with their Conservative values.
• Experiments / evaluation:
  • Participants: 465 from India and the US in Phase 1 and 2; 180 additional participants in Phase 3.
  • Metrics: Schwartz’s value categories, linguistic constructiveness features, and perceived value alignment.
  • Topics: Homophobia and Islamophobia, chosen for their divisiveness and value-laden nature.
• Limitations and future work:
  • Limited to two cultures (India, US) and two topics (homophobia, Islamophobia); findings may not generalize to other contexts.
  • Focused on GPT-4; future work should explore other LLMs and multilingual settings.
  • Non-representative samples due to recruitment constraints (Prolific’s Harmful Content Prescreener).

Summary

This study demonstrates that LLMs systematically homogenize values when rewriting comments on value-laden topics, downplaying Conservative values and emphasizing prosocial ones like Universalism and Benevolence. While this enhances linguistic constructiveness and aligns with progressive viewpoints, it risks marginalizing Conservative perspectives and altering stances. Cross-cultural experiments reveal that LLM-rewritten comments resonate more with participants supporting same-sex marriage or Islam but alienate those opposing these topics. These findings highlight the ethical and socio-political challenges of deploying LLMs in sensitive discourse, underscoring the need for transparency, personalization, and user control in AI-mediated communication.