Publication bias in randomized trials of psilocybin for depression: a Robust Bayesian Meta-Analysis

A re-analysis of 12 randomized controlled trials on psilocybin for depression found moderate evidence of publication bias, with a bias-corrected effect size about 41% smaller than originally reported. The original analysis reported a large effect (g = 0.90), but after accounting for selective reporting using Robust Bayesian Meta-Analysis, the effect dropped to about 0.53, with a credible interval that includes zero. This evidence of bias disappeared when open-label studies were excluded, suggesting expectancy-driven effects rather than suppressed negative results. The findings do not overturn the original conclusions but indicate greater uncertainty than previously thought.

Study at a glance

Abstract

A recent living systematic review in Nature Mental Health found psilocybin reduced depression symptoms across 12 randomized controlled trials, with pooled effect g = 0.90. The original analysis assessed publication bias using Egger’s test, non-significant — but this test has limited power with few studies and cannot produce a bias-corrected estimate. We re-analysed the same open dataset using Robust Bayesian Meta-Analysis (RoBMA), which simultaneously estimates evidence for a treatment effect, heterogeneity, and publication bias. We find moderate evidence for publication bias (Bayes Factor = 3.89). After accounting for selective reporting, the bias-corrected effect (g ≈ 0.53) is ~41% smaller than published, with a credible interval spanning to zero. This evidence dissipates when open-label studies are excluded, suggesting expectancy-driven artefacts rather than file-drawer suppression. These findings do not overturn the original conclusions but indicate greater uncertainty than the frequentist analysis implies.