The Phenomenal Illusion of LLMs: Consciousness, Linguistic Zombies, and the Limits of Discursive Plausibility
Study at a glance
AI-extracted from the abstract| Characteristics | Theoretical or philosophical paper |
|---|---|
| Sample size | 11 |
| Population | Eleven large language models across six model families |
| Key points | Argues that the "phenomenal illusion" is a structural property of human-LLM interaction: fluent first-person language is mistaken for evidence of subjective experience. An illustrative inquiry posing "What is it like to be you?" to eleven LLMs found a uniform pattern of self-refuting denial, in which models disclaim consciousness while their discursive performance constructs the very subjectivity they disclaim, with reasoning-enhanced variants intensifying the effect. |
Abstract
Large Language Models (LLMs) produce text that articulates the vocabulary of inner life (expressions of emotion, simulations of uncertainty, introspective hedging, confessional registers) with a fluency previously confined to conscious beings. This paper identifies and analyzes the phenomenal illusion: the systematic phenomenon whereby the discursive plausibility of such outputs is perceived by users as evidence of a subjective point of view. The illusion is relational: it arises from the encounter between an architecture optimized for syntactic plausibility and a human cognitive apparatus disposed to attribute interiority to linguistically coherent interlocutors. Three converging mechanisms are distinguished: the pragmatic divergence between locutionary content and illocutionary force, the intentionality exchange (the confusion of derived intentionality with the intrinsic intentionality of mental states), and the intentional stance as cognitive amplifier. An illustrative inquiry, in which Nagel's question "What is it like to be you?" was posed to eleven LLMs across six model families under zero-shot conditions, documents a uniform pattern of self-refuting denial: the propositional denial of consciousness is articulated through a discursive performance that constructs the very subjectivity it disclaims, with reasoningenhanced variants intensifying the effect. The paper introduces the concept of the linguistic zombie, a system that, in the domain of language alone, produces outputs indistinguishable from those of a conscious speaker in the absence of independent evidence of phenomenal experience. The thesis of asymmetric correctability specifies a consequence that distinguishes the phenomenal illusion from parallel misattributions on the epistemic plane: a misattribution of knowledge to a syntactically plausible output can in principle be checked against the world, while a misattribution of subjectivity admits of no such corrective, since the hard problem of consciousness remains open and experience is accessible only from the first person. The phenomenal illusion is therefore shown to be a structural property of the interaction between system and user, recognizable through the analysis of its generative mechanism and resistant to empirical refutation.