Computational Dualism and Objective Superintelligence
Study at a glance
AI-extracted from the abstract| Characteristics | Theoretical or philosophical paper |
|---|---|
| Key points | Argues that claims about software superintelligence rest on a flawed "computational dualism" that separates software from the hardware interpreting it, and proposes a pancomputational account in which cognition is embodied, embedded, extended and enactive. On this account intelligence is the ability to "generalise", identify causes and adapt, and the authors derive upper bounds suggesting AGI will be safer but more limited than theorised. |
Abstract
The concept of intelligent software is flawed. The behaviour of software is determined by the hardware that "interprets" it. This undermines claims regarding the behaviour of theorised, software superintelligence. Here we characterise this problem as "computational dualism", where instead of mental and physical substance, we have software and hardware. We argue that to make objective claims regarding performance we must avoid computational dualism. We propose a pancomputational alternative wherein every aspect of the environment is a relation between irreducible states. We formalise systems as behaviour (inputs and outputs), and cognition as embodied, embedded, extended and enactive. The result is cognition formalised as a part of the environment, rather than as a disembodied policy interacting with the environment through an interpreter. This allows us to make objective claims regarding intelligence, which we argue is the ability to "generalise", identify causes and adapt. We then establish objective upper bounds for intelligent behaviour. This suggests AGI will be safer, but more limited, than theorised.