Selection bias can inflate the Perturbational Complexity Index (PCIst) in high-dimensional low-SNR regimes: diagnosis, boundary conditions, and a fully out-of-sample estimator

Abstract

The state-transition Perturbational Complexity Index (PCIst; Comolatti et al., 2019, Brain Stimulation) is the most practical validated measure of consciousness-related brain complexity, discriminating wakefulness from sleep, anesthesia and disorders of consciousness from the complexity of TMS-evoked responses. Its pipeline, however, performs three data-dependent selections — the SVD basis is fit on the response window it later quantifies, principal components are retained by an SNR criterion evaluated on the same data, and per-component state-transition thresholds are chosen by maximization on the same data. We show in simulation that these nested selections can produce large PCIst values in the complete absence of any response to the perturbation. In recurrent network models with weakly correlated per-node noise, sham (no-pulse) data yields PCIst ≈ 500, (v4) of the same magnitude as pulsed data (three networks per condition; v3 wrote "statistically indistinguishable", for which no test was performed and none is possible at that n); retained components exhibit SNRs narrowly hugging the acceptance threshold (1.13–1.51 for a cutoff of 1.1), and the inflation is invariant to trial count (n = 10–160), because the SNR criterion is a ratio of the same trial-averaged signal. Under realistic clinical conditions (60 channels, 1/f + alpha spatially correlated noise, standard windows), inflation is modest (sham ≈ 6–12 vs ≈ 45 for genuine weak responses) and classifications remain robust — but sham inflation grows monotonically with the effective dimensionality of the noise (Spearman ρ = 0.72, p < 1e-39), reaching PCIst > 100 for independent-channel noise, with heavy-tailed outliers (single sham realizations up to 210) even in low-dimensional regimes. This defines a danger zone directly relevant to proposed extensions of PCI to microelectrode arrays, organoids, animal preparations and artificial systems. We introduce PCIst-XV, a fully out-of-sample estimator (basis, component selection and thresholds fitted on half the trials; transitions counted on the held-out half) that reduces sham values to near floor in all tested regimes while matching or exceeding the standard estimator's discrimination of weak real responses (AUC 1.00 vs 0.95–1.00). An independent reimplementation from the published equations (sharing no code with the GPL reference implementation) reproduces the reference implementation exactly (r = 1.0000, values (v4) identical to machine precision — maximum absolute difference 1.1 × 10⁻¹³ over 16 test cases; v3 understated this as "identical to one decimal". The 16 cases were run in one adversarial regime (spectral radius 0.3); equivalence under clinical-SNR conditions was not tested), establishing the bias as a property of the method, not its code. On real human TMS-EEG (Biabani et al., 123 subjects, two independent samples), genuine TEPs yield PCIst in the published range (median 34.3/26.9) while matched pre-stimulus nulls stay low (median 2.6/3.1) — as our danger-zone analysis predicts, since real trial-averaged scalp EEG has effective dimensionality 1.0–4.9, squarely in the safe zone; raising min_snr to 2.0, however, cuts genuine TEP values by 38–45% at the median on real data, a substantially higher sensitivity cost than simulation suggests. We recommend sham-control and/or cross-validation whenever PCIst is applied outside its validated clinical domain.