How Robustly Can Integrated Information Be Estimated in Artificial Neural Networks?

Abstract

Integrated Information Theory (IIT) identifies consciousness with the intrinsic, irreducible cause–effect power of a physical system, quantified by integrated information Φ, and is therefore often applied to artificial neural networks (ANNs) via scalable proxies. Our central finding is that such proxies are not currently reliable enough to support the comparisons they are used for. We test one such proxy, the Gaussian Φ*, rather than the family of candidate Φ measures, and our conclusions are scoped accordingly. Estimating Φ* (Barrett–Seth / Oizumi–Amari) across RNN, GRU and LSTM networks, we find that which architecture appears most integrated is determined by the covariance regularization the analyst chooses: a pipeline ablation attributes a full reversal of the architecture ranking almost entirely to one correction, the numerical stabilization of a rank-deficient covariance, and a sensitivity analysis shows this single knob moves Φ* by more than the entire gap between architectures. A synthetic validation with known ground truth shows Φ* tracks coupling only above a strength of about 0.10, has a non-zero floor below that, and conflates integration with redundancy. Choosing the regularization analytically rather than by convention does not resolve this: the optimal intensity is itself architecture dependent, so adopting it per system reintroduces the very unmatched-settings confound that corrupts such comparisons. We reach this through a matched re-implementation of our own earlier Φ* pipeline, which suffered from unmatched estimator settings, near-singular covariances producing unrealistically large mutual-information values, and single-run evaluation. Re-implemented as a single matched, numerically stabilized, five-seed estimator, the ranking reverses, with the RNN ranking first at every shrinkage value tested (one-way ANOVA F(2,12) = 25.7, p = 4.6 × 10⁻⁵) and the GRU and LSTM statistically indistinguishable. We treat that ordering as a property of the matched estimator rather than as evidence that the RNN is intrinsically more integrated. Task accuracy is a weak predictor of Φ*. We also compute exact Φ (IIT 3.0, via PyPhi) for small systems, where Φ = 0 for a feedforward classifier is entailed by the formalism rather than discovered, and use it only to validate our pipeline. We do not claim to have established a dissociation between computation and integration: that would require input–output-equivalent systems, which we identify as the key open experiment. We make no claim about machine consciousness; the contribution is a reproducible, critical account of the limits of operationalizing IIT in artificial systems.