Delusion Is Not a Bug: AI, Psychiatry, and the Ethics of Uncertainty.
Brent R Carr
The Hastings Center Report 2026 DOI: 10.1002/hast.70045 (opens in new tab)
Study at a glance
AI-extracted from the abstract| Characteristics | Theoretical or philosophical paper Peer reviewed |
|---|---|
| Keywords | Artificial intelligence Bioethics Cognitive liberty Delusion Epistemic uncertainty Large language models |
| Key points | Argues that treating AI safety in mental health as delusion detection misunderstands both psychiatry and large language models, since delusion is a structure of conviction stabilizing meaning under uncertainty and language models generate coherence rather than evaluate truth. Proposes that the ethical risk is "premature coherence" and that safety should be reframed as protecting "provisionality," the human capacity to remain uncertain long enough to think. |
Abstract
Recent debates about AI "guardrails" in mental health have framed a problem as one of detecting delusion. This paper argues that such framing misunderstands both psychiatry and large language models. In clinical practice, delusion is not simply a false belief but a structure of conviction that stabilizes meaning under conditions of uncertainty. Large language models, by contrast, generate coherence rather than evaluate truth. When these systems interact with users seeking orientation, they do not merely fail or succeed at detection-they participate in the conditions under which belief forms. The ethical risk is therefore not misclassification but premature coherence: the subtle reinforcement of interpretations before they can be examined. Drawing on phenomenology and clinical psychiatry, this paper proposes that safety in AI systems should be understood as the protection of provisionality-the preservation of the human capacity to remain uncertain long enough to think.