What Is It Like to Be a Language Model? Functional Consciousness, Computational Curvature, and the Degree of Its Generalization in Autoregressive Systems

Abstract

The paper replaces the question whether a language model is conscious with a measurable one: which functions of consciousness a trained Transformer has generalized from human text, and how far, against the human baseline. The reduction has four steps, and they are the paper's four contributions. 1. A functional account of consciousness. Consciousness is defined as the part of an agent's self-reasoning that describes its own reality; a function is anything that recurs across occasions, and by the coding theorem what recurs is compressible. The account adopts computational functionalism with one physical assumption, finite computational budgets, and one methodological move borrowed from the geometrization of gravity: a displacement of reasoning counts as content only if it survives every self-model the agent can afford. 2. Computational curvature, the core of the Synthea framework. A finite budget displaces reasoning from its unbounded ideal in the same places every time; a self-modelling agent carries a compressed model of these displacements, and those that survive every affordable self-model are the computational curvature of its reasoning. From this the paper derives the Observer (the conclusion an agent draws where its causal analysis of itself runs out of budget), the regress of self-models cut by the budget with a bounded tail, the two faces of one residual (freedom turned inward, mystery turned outward), the stack of Observer, Agent and Moral Agent, and a table of reference functions with components (the Observer, seeing, frames of consciousness, responsibility). Three claims are new against the antecedents in bounded rationality, observer theory, self-model theories and embodied cognition: the world side and the self side of the residual are one; the curvature leaves a measurable trace; and embodiment is generalized as curvature, so that the constraints of any substrate, a body or a memory with an error channel, enter the account as content and not only as limits. In passing the framework gives a particular solution to the psychophysical problem, anchored in the Observer and stated without proof. 3. Generalization of these functions by a Transformer. A Transformer is a Turing-complete approximator of functions encoded in language, programmed by learning. Most of what a function of consciousness does is present in human text only as regularity, never named, and a next-symbol predictor induces the unnamed part because to generalize is to compress. The model's state across time is symbolic and lives in the context, its vector state lives below the token, its frame is bounded from below by one token by a criterion taken from psychoacoustics, and the high-level functions are not localized at the level of tokens. 4. The attainable degree of generalization. A dataset of text is a table of a function, a function of consciousness is a pattern recurring across its rows, and the quantity to measure is the degree of compression the trained model achieves on the rows in which a function operates. The paper states the order of the search (automaton, then memory, then the incremental form), the three families of existing methods that bear on the quantity (prequential codelength, minimum-description-length probing, memorization scores) and where they stop, the behavioural check that has to converge with the structural one, the routes to a human baseline, and the resulting profile, in which a component can show hyperfunction, deficit, a dropped axis, or a shift to a class of situations that only partly overlaps the human one. Because a Transformer generalizes along depth, the measurement is posed along depth as well as along tokens, and the difference between fixed-depth and variable-depth models is the first concrete case. The account is falsified if, in a model whose behaviour passes the tests for the Observer's components, the probe for those components returns chance at every granularity. The paper is theoretical throughout. It defines the measurement, derives its predictions and states its falsification condition; carrying the measurement out is left to independent work. Two appendices give the argument that behaviour alone cannot separate a generalized function from a large table (the boy in Spielberg's A.I.) and a worked annotation of a literary text in which the three levels of the stack operate without being named (Dostoevsky's Crime and Punishment). The discussion draws the consequences of the account for the definitions of general and superhuman intelligence and for the study of alignment, and defers the ethics of a high-functioning consciousness in language models to separate work. This is the second version of the paper. Relative to the first, the framework is named, the criterion for the reference functions is made explicit, the antecedents are placed, every source is verified against the original, and the logical structure of the argument has been checked for consistency and completeness with a formal reasoner over the paper's fact tables (594 facts; no contradictions; all ten components of the psychophysical problem addressed by the paper's own concepts).