Reasonable expectations for replications of psychedelic-assisted therapy: The case for prediction intervals rather than P-values
Mitch Earleywine, Philip Kamilar-Britt
Journal of Psychedelic Studies January 16, 2023 DOI: 10.1556/2054.2022.00219 (opens in new tab)
Study at a glance
AI-extracted from the abstract| Characteristics | Review Peer reviewed |
|---|---|
| Topics | Psychedelic-assisted therapy |
| Key findings | Trials of psychedelic-assisted therapy that emphasize statistical significance will likely include failures to replicate unless sample sizes increase dramatically, but prediction intervals show that reasonable replications are within reach. |
Abstract
AbstractPsychedelic compounds hold promise for alleviating human suffering. Initial trials of psychedelic-assisted treatments have established feasibility and safety, generating calls for replications. Meanwhile, social and medical sciences have drawn criticism due to perceptions of replication failures and varying public trust in empiricism. Data suggest that researchers and the public frequently misunderstand some of the statistical issues associated with replication, potentially leading to unrealistic expectations of treatment effects. Promoting discourse on what constitutes sufficient replication is especially warranted considering the ongoing progression of multi-site phase II and III clinical trials. Here, we review recent and classic work on prediction intervals and power analysis to reveal that trials of psychedelic-assisted therapy that emphasize statistical significance will likely include failures to replicate, especially if sample sizes do not increase dramatically. The field and the public should expect some failed replication attempts based on sampling variability alone. Continued emphasis on statistical significance will require markedly larger samples than those used in clinical trials to date, necessitating substantially greater resources. An alternative approach focused on prediction intervals has distinct advantages. We focus on a recent trial of MDMA-assisted therapy for PTSD to show that, based on prediction intervals, reasonable replications are well within reach. A lack of attention to these statistical issues could unnecessarily prompt widespread dismissal of these therapies before the intervention receives adequate investigation and a fair assessment. In contrast, realistic expectations and appropriate planning could help ensure that these treatments receive the opportunity to help those most in need.
Comparable studies
Other narrative reviews on psychedelic-assisted therapy, most cited first.
Citations in the library
Cited by 3
- The Therapeutic Alliance in Psychedelic-Assisted Psychotherapy: A Novel Target for Research and Interventions Psychedelic Medicine August 18, 2023
- Perceptions of Psilocybin-Assisted Psychotherapy and Standard Interventions for Nicotine Cessation Journal of Psychoactive Drugs 2026
- Methodological fragmentation and integration priorities in psychedelic-assisted therapy assessment: a scoping review Frontiers in Psychiatry August 6, 2026
Cites 5
- MDMA-assisted therapy for severe PTSD: a randomized, double-blind, placebo-controlled phase 3 study. Nature Medicine June 1, 2021
- A Meta-Analysis of Placebo-Controlled Trials of Psychedelic-Assisted Therapy Journal of Psychoactive Drugs June 12, 2020
- Efficacy of psychedelic treatments on depressive symptoms: A meta-analysis Journal of Psychopharmacology May 25, 2020
- A comparison of MDMA-assisted psychotherapy to non-assisted psychotherapy in treatment-resistant PTSD: A systematic review and meta-analysis Journal of Psychopharmacology December 20, 2020
- Psychedelic-assisted psychotherapy for depression: How dire is the need? How could we do it? Journal of Psychedelic Studies June 1, 2020