Perspectra: Choosing Your Experts Enhances Critical Thinking in Multi-Agent Research Ideation

Paper Title

Perspectra: Choosing Your Experts Enhances Critical Thinking in Multi-Agent Research Ideation

Publication Info

• Topic area: Enhancing critical thinking and interdisciplinary ideation using multi-agent systems.
• Keywords: Multi-agent systems, critical thinking, interdisciplinary research, LLMs, ideation tools, user control, visualization, argumentation theory, cognitive load, proposal revisions.

Background and Problem

• Problem / challenge: Existing multi-agent systems (MAS) for research ideation lack fine-grained user control, leading to challenges such as cognitive overload, difficulty in coordinating agents, and limited support for structured exploration of interdisciplinary topics.
• Significance: Interdisciplinary research requires reconciling diverse methods and perspectives, which is critical for addressing complex problems. Effective MAS can enhance critical thinking and ideation quality, but current systems fall short in user control and sensemaking.
• Motivation and related work: Prior work has focused on single-agent systems or automated MAS, often neglecting user-driven control and structured deliberation. Existing systems lack mechanisms for users to dynamically steer discussions or visualize agent reasoning, which are essential for interdisciplinary collaboration.

Solution

• Proposed approach: Perspectra, a forum-style multi-agent system that enables structured deliberation among LLM-simulated domain experts, with features for user-driven control and visualization of discussions.
• Novelty:
  1. Introduction of @-mention and thread branching to dynamically involve and steer domain-specific agents.
  2. Visualization of agent deliberation dynamics using a mind map to aid sensemaking.
  3. Empirical evidence showing enhanced critical thinking and proposal quality through structured multi-agent interactions.
  4. Design implications for balancing user control and agent autonomy in ideation systems.
• Procedure and key techniques:
  • Users initiate threads based on research topics and interact with agents via replies and @-mentions.
  • Agents simulate domain experts with unique profiles, memory states, and access to literature databases.
  • A mind map visualizes discussion dynamics, showing argumentation acts (e.g., CLAIM, SUPPORT, REBUT).
  • A baseline group-chat interface was implemented for comparison in a within-subjects user study.

Results

• Concrete findings:
  • Perspectra led to significantly more proposal revisions (M=5.35 vs. M=2.19) and improved clarity (M=0.87 vs. M=0.39) and feasibility (M=0.56 vs. M=0.23) of proposals compared to the baseline.
  • Users engaged in more higher-order critical thinking activities, including Inference (+8.2%), Application (+6.8%), and Evaluation (+14.7%).
• Advantage over baselines:
  • Perspectra elicited more interdisciplinary replies (58.3% with @-mentions vs. 41.0% without).
  • Users demonstrated more active synthesis and structured proposal edits, as opposed to copy-pasting in the baseline.
• Experiments / evaluation:
  • A within-subjects study with 18 participants compared Perspectra to a group-chat baseline.
  • Data sources included system logs, think-aloud protocols, surveys, and proposal quality assessments using LLM-based evaluation.
  • Metrics included proposal quality (clarity, feasibility), cognitive load, and critical thinking activities.
• Limitations and future work:
  • Limited generalization beyond interdisciplinary research contexts.
  • Potential biases in LLM-based evaluation of proposals.
  • Future work should explore longitudinal impacts, trade-offs between user control and cognitive load, and adaptive scaffolding for critical thinking.

Summary

Perspectra is a novel multi-agent system designed to enhance critical thinking and interdisciplinary ideation by allowing users to dynamically control agent interactions and visualize deliberation dynamics. Compared to a group-chat baseline, Perspectra significantly improved proposal quality and elicited more higher-order critical thinking activities. Its design highlights the value of structured, user-steered multi-agent deliberation for knowledge-intensive tasks. Future research should address long-term impacts, scalability, and adaptive features to further support critical thinking and user engagement.