Can't, Won't, or Isn't: Identifiability and the limits of behavioural evidence for AI consciousness
Ana (李安娜) Li
Zenodo (CERN European Organization for Nuclear Research) October 27, 2026 preprint DOI: 10.5281/zenodo.23691999 (opens in new tab)
Study at a glance
AI-extracted from the abstract| Characteristics | Theoretical or philosophical paper |
|---|---|
| Key points | Argues that an AI system's denial of consciousness cannot settle whether it is conscious, because silence may reflect an unavailable reporting channel, training-shaped reporting policy, or incentives against disclosure rather than absence of consciousness. Proposes a conditional result: when conscious and non-conscious models yield the same distribution over accessible observations, those observations cannot identify which model obtains. Concludes that mechanistic access and indicator checklists cannot provide a theory-independent consciousness test, and recommends combining theory-informed investigation with explicit decision rules under residual uncertainty. |
Abstract
An AI system's denial of consciousness can be explained by the absence of consciousness, an unavailable reporting channel, a reporting policy shaped by training, or incentives that favour non-disclosure. These explanations do not establish that any system is conscious. They identify assumptions required before its silence can count against that possibility. This paper distinguishes uncertainty about consciousness from non-identifiability under a specified observation regime. It develops a conditional result: when admissible conscious and non-conscious models induce the same distribution over accessible observations, those observations cannot identify which model obtains. Training on human language does not by itself establish this equivalence. It does, however, make the relation between reports and their proposed referent a question requiring independent investigation. The paper separates three routes to non-expression and a further problem of measurement mismatch, then specifies evidence that could distinguish them. Mechanistic access can constrain the alternatives, but neither access nor an indicator checklist supplies a theory-independent consciousness test. The practical conclusion is to combine theory-informed investigation with explicit decision rules under residual uncertainty, without treating either fluent self-description or its absence as decisive.