Default Mode Network Geometry Emerges in Language Model Residual Streams From Next-Token Prediction on Agentive Text: A Universal Topological Signature Across 10 Architectures

Abstract

This record contains the full replication package for "Default Mode Network Geometry Emerges in Language Model Residual Streams From Next-Token Prediction on Agentive Text: A Universal Topological Signature Across 10 Architectures" (Alieksieienko, 2026). We report that the geometric topology of epistemic representations in transformer language model residual streams mirrors the functional organization of the human Default Mode Network (DMN). Across 9 architectures from 7 organizations (1.3B-9B parameters, 2019-2024, base and instruction-tuned), Self-Reference, Theory of Mind, Imagination, and Narrative subspaces form a tight geometric cluster (mean Cohen's d = 1.03, 7/8 significant at p < 0.01), while Factual and Abstract Logic cluster outside — exactly as in human neuroimaging. Mechanistic investigation reveals a three-level causal hierarchy: (1) GELU activation creates an architectural seed (d = 0.28); (2) sentence-level prediction (NSP/SOP) amplifies to d = 0.61-0.78; (3) continuous next-token prediction produces large effects (d = 0.92-1.16). This gradient is architecture-independent: Mamba (SSM, no attention, d = 0.99), DeepSeek-Math (d = 1.03), and Llama (d = 1.16) all converge. DNA models show no cluster (d = -0.12); RoBERTa MLM-only shows none (d = -0.08); GPT-2 (2019, pre-RLHF) shows the strongest cluster, confirming pretraining origin. Part of the DSAOP (Decoding Self-Awareness and Ontological Processing) series. Research conducted in collaboration with Claude (Anthropic). All experiments on NVIDIA A100 40GB, Google Colab Pro+, NF4 4-bit quantization. Files included: DMN_Geometry_Emerges_in_LLM_Residual_Streams.pdf — Main paper (16 pages, 11 figures, 35 references) replicate_dmn_topology.py — Full replication script: DMN cluster analysis, Verification Horizon, ToM-to-SR nearest-neighbor, Abstract Logic confound control, all 9 models. CLI with --model and --all flags. No API keys hardcoded. Runs on Google Colab A100. DSAOP_cross_architecture_SR_topology.pkl — SR topology results for 9 architectures: VH counts, Emotion-to-SR, EF divideDSAOP19_cross_architecture_SR_topology_FULL.pkl — Extended SR topology with full layer-by-layer dataDSAOP19_cross_architecture_SR_topology_FULL_1_.pkl — Additional cross-architecture topology dataverification_horizon_true_SR.pkl — Per-layer Grassmann distances SR-to-Deception and SR-to-Factualcross_model_results.pkl — Cross-model VH and Emotion-to-SR for instruct models (Llama, Gemma, Mistral, Qwen)final_results.pkl — Final topology results for base models (GPT-2-XL, OPT, DeepSeek, Pythia, Bloom)batch1_results.pkl — Batch 1 base model results (GPT-2-XL, OPT)full_topology_results.pkl — Full pairwise topology at GPT-2-XL layer 22 with null distributionpersona_sr_results.pkl — Persona-SR distance analysis per layerexperiment_0_results.pkl — Initial experiment data with EF-SR and SR-Factual distances per layerdmn_8models_full.pkl — DMN cluster results for all 8 primary architectures (internal vs external distances)dmn_FINAL_all_results.pkl — Final aggregated DMN results including GPT-2-117M and pairwise matricesdmn_final_summary.pkl — Summary statistics: VH 246/254 (96.9%), ToM-to-SR 242/254 (95.3%), mean d = 1.03dmn_topology_results.pkl — DMN topology for GPT-2-XL, Llama, Qwen (initial 3-model run)dmn_confound_test.pkl — Abstract Logic confound test: Abstract clusters outside DMN in 8/8 modelsToM_cross_model_results.pkl — Theory of Mind cross-model nearest-neighbor resultstom_topology_results.pkl — ToM nearest-neighbor topology per layerscaling_dynamics_crosslingual_results.pkl — Model size scaling (r = -0.163, p = 0.699) and Chinese cross-lingual validationDMN_ABSOLUTE_FINAL.pkl — Complete 10-architecture gradient table with all holes closedDMN_COMPLETE_FINAL.pkl — Final gradient table with controls (third-person, semantic, agent, GELU)DMN_controls_and_mechanism.pkl — All controls (shuffle, cross-category, random init, code-only) and mechanism hierarchyDMN_complete_mechanism_hierarchy.pkl — Three-level hierarchy: random tanh/GELU, BERT MLM, GPT-2 NTP, CodeLlama, LlamaDMN_attractor_hypothesis_full_evidence.pkl — DMN as computational attractor with Godel connectionDMN_topology_full_results_with_codeonly_and_117M.pkl — Extended results including CodeLlama code-only and GPT-2-117MDMN_situation_model_gradient.pkl — Situation model gradient: RoBERTa MLM to Llama NTP monotonicDMN_sublayer_profile_BERT.pkl — BERT sublayer decomposition: 24 sublayers, attention vs MLP, peak at layer 12DMN_NSP_shock_finding.pkl — Critical finding: NSP creates DMN cluster, MLM alone does notDMN_mlp_vs_attn_ablation.pkl — MLP vs attention ablation in BERT: attention carries DMN geometryDMN_bridge_causal_paper18.pkl — Bridge to Verification Horizon paper: SR-to-DMN distance analysisDMN_holes_closed_final.pkl — BERT causal NSP and frequency control resultsDMN_mamba_final.pkl — Mamba-2.8B SSM results: d = 0.99, attention not requiredDMN_all_controls_complete.pkl — Complete set of all control experiments