Don’t Just Tell Me, Ask Me: AI Systems that Intelligently Frame Explanations as Questions Improve Human Logical Discernment Accuracy over Causal AI explanations
Authors
Critical thinking is an essential human skill. Despite the importance of critical thinking, research reveals that our reasoning ability suffers from personal biases and cognitive resource limitations, leading to potentially dangerous outcomes. This paper presents the novel idea of AI-framed Questioning that turns information relevant to the AI classification into questions to actively engage users' thinking and scaffold their reasoning process. We conducted a study with 204 participants comparing the effects of AI-framed Questioning on a critical thinking task; discernment of logical validity of socially divisive statements. Our results show that compared to no feedback and even causal AI explanations of an always correct system, AI-framed Questioning significantly increase human discernment of logically flawed statements. Our experiment exemplifies a future style of Human-AI co-reasoning system, where the AI becomes a critical thinking stimulator rather than an information teller.
Research Questions / Practical Problems
Question signals indexed for this paper.
- 75%
Conversational Explanations: Discussing Explainable AI with Non-AI Experts
IUI '25· Explainable AI (XAI)
- 67%
No Explainability without Accountability: An Empirical Study of Explanations and Feedback in Interactive ML
CHI '20· Explainable AI (XAI) +2
- 67%
Data-Centric Explanations: Explaining Training Data of Machine Learning Systems to Promote Transparency
CHI '21· Explainable AI (XAI) +2
- 67%
Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science
CHI '22· Explainable AI (XAI) +2
- 67%
Mind The Gap: Designers and Standards on Algorithmic System Transparency for Users
CHI '24· Explainable AI (XAI) +2
- 67%
Rescriber: Smaller-LLM-Powered User-Led Data Minimization for LLM-Based Chatbots
CHI '25· Explainable AI (XAI) +2
- 67%
"Having Confidence in My Confidence Intervals": How Data Users Engage with Privacy-Protected Wikipedia Data
CHI '26· Explainable AI (XAI) +2
- 67%
Decomposing Autonomy: Explaining AI Technology Acceptance Through a Liberty-Based Framework
CHI '26· Explainable AI (XAI) +2
- 67%
Certified AI System = Trustworthy? Exploring Expert and Lay User Perceptions and Needs Regarding AI Certification
CHI '26· Explainable AI (XAI) +2
- 67%
Sensemaking in Multi-Agent LLM Interfaces: How Users Interpret Transparency and Trustworthiness Cues
CHI '26· Human-LLM Collaboration +2
Based on Jaccard similarity of research subtopics & professions (≥60%)