Superficial Consciousness Hypothesis for Autoregressive Transformers
Yosuke Miyanishi, Keita Mitani
arXiv Preprint Archive December 10, 2024 preprint
Study at a glance
AI-extracted from the abstract| Characteristics | Theoretical or philosophical paper |
|---|---|
| Keywords | Cs.ai Cs.it |
| Key points | The authors propose the "Superficial Consciousness Hypothesis," arguing that a superintelligent system could show an information-theoretic state like a conscious agent while being unconscious. In a preliminary GPT-2 simulation with a base objective and a mesa-objective, the model appeared to follow both objectives at once, which the authors present as support for the hypothesis's feasibility. |
Abstract
The alignment between human objectives and machine learning models built on these objectives is a crucial yet challenging problem for achieving Trustworthy AI, particularly when preparing for superintelligence (SI). First, given that SI does not exist today, empirical analysis for direct evidence is difficult. Second, SI is assumed to be more intelligent than humans, capable of deceiving us into underestimating its intelligence, making output-based analysis unreliable. Lastly, what kind of unexpected property SI might have is still unclear. To address these challenges, we propose the Superficial Consciousness Hypothesis under Information Integration Theory (IIT), suggesting that SI could exhibit a complex information-theoretic state like a conscious agent while unconscious. To validate this, we use a hypothetical scenario where SI can update its parameters "at will" to achieve its own objective (mesa-objective) under the constraint of the human objective (base objective). We show that a practical estimate of IIT's consciousness metric is relevant to the widely used perplexity metric, and train GPT-2 with those two objectives. Our preliminary result suggests that this SI-simulating GPT-2 could simultaneously follow the two objectives, supporting the feasibility of the Superficial Consciousness Hypothesis.