Editorial: Open science in consciousness research

Abstract

All of us involved in the mind and brain sciences, in whatever capacity, are increasingly aware of the importance of ensuring the credibility of our research. This credibility—which for experimental work turns on its reliability—is particularly salient for consciousness research, given the at-times precarious perception of consciousness science within the wider landscapes of psychology and neuroscience (Michel et al. 2019). We are therefore very pleased to introduce a number of ‘open science’ initiatives that have recently been implemented in Neuroscience of Consciousness. One key objective of open science is to actively resist the tendency to search for, and to preferentially publish, eye-catching findings that fit with compelling narratives. This means supporting studies that have been designed to report unbiased results that are reproducible and replicable, whether or not they are narratively convenient. The gold standard approach here is the Registered Report (RR) format. Intuitively, the idea is to allow researchers to retain complete control over their hypotheses, methods and interpretations, but not over the results (Hardwicke and Ioannidis 2018; Chambers 2019). RRs divide the peer review process into two stages. In the first, reviewers evaluate the rationale, hypotheses, methods and proposed analyses for a study. These are described in a ‘stage 1’ manuscript. This stage takes place before any data collection, other than pilot data collected purely to inform study design. Submissions can be rejected, sent for revision, or accepted at this stage. Importantly, stage 1 peer review offers opportunities to enhance study design—including correction of potentially fatal confounds and flaws—while there is still time. This benefits the researchers and is rewarding for the reviewers. Once a stage 1 submission is accepted, it is preregistered in a time-stamped public repository such as the Open Science Framework (https://osf.io/), with or without a temporary embargo. Following this ‘in principle acceptance’, the researchers carry out their planned data collection and perform their preregistered analyses, as well as any additional, clearly demarcated, exploratory analyses. They then submit a stage 2 manuscript incorporating these results and a discussion. The introduction and methods remain unchanged from the stage 1 in-principle-accepted manuscript. This stage 2 submission is sent back to the original reviewers—who cannot at this point raise post hoc objections to the study rationale or design. Critically, the journal is committed to publish this stage 2 manuscript no matter how the results turn out, so long as the researchers followed the procedures that they described in their stage 1 submission, and have written up their results and discussion sections satisfactorily, as judged by the reviewers. The wow-factor of the results is simply not relevant to the acceptance decision. In this way, publication of a RR does not depend on the appeal of a post hoc story based on eye-catching findings, but instead on the significance of the research question and the adequacy of the experimental design to address this question. As mentioned, exploratory analyses are not excluded by the RR format. In fact, they are encouraged—so long as they are clearly marked as exploratory and reported separately from the primary preregistered analyses. Neuroscience of Consciousness has been accepting RR submissions since June 2019, and we are delighted that Prof. Zoltán Dienes (University of Sussex, UK) has joined our editorial team specifically to handle these manuscripts. We strongly encourage consciousness researchers to take advantage of this innovative format. The journal webpages contain further information about the format, including detailed instructions on how to prepare RR submissions (https://academic.oup.com/nc/pages/ General_Instructions). Despite the many benefits offered by RRs, both to individual researchers and to the wider community, we recognize that it is not always possible to follow this process to its fullest extent. This may be the case when data collection cannot be timed to the researcher’s convenience; where the research is purely exploratory, or focuses on methodological innovations, or is otherwise constrained in ways that preclude the RR procedure. Some kinds of research—such as computational modelling—fit uneasily in present RR formats. Fortunately, RRs are not the only tool by which open science can be facilitated. Experimental designs and proposed analyses can still be—and wherever possible should be—preregistered prior to data collection (Nosek et al., 2018). This means that the critical distinction between (preregistered) confirmatory and (non-preregistered) exploratory analyses can still be maintained. As with RRs, this separation mitigates against the selective reporting or ‘cherry picking’ of results, as well as against p-hacking (conducting repeated analyses until a desired outcome is obtained, without sufficient statistical correction) and HARK-ing (hypothesizing-after-the-results-are-known)—all of which constitute seductive traps for non-preregistered research. Several options for preregistration are available. One flexible format is provided by www.aspredicted.org—with the Open Science Framework again providing a suitable repository. Other open science initiatives focus on transparency. Experimental data can be shared, along with code