Registered Reports: one policy-level fix (with real trade-offs)
Last week we named three barriers to implementing open science practices in one’s research: time, fear of being scooped, and unclear incentives. This week, we take a look at Registered Reports, which represent one structural attempt to address at least one of these challenges: namely, incentives.
The mechanism is simple in principle: authors submit their research question, methods, and analysis plan to a journal before collecting any data. That protocol is peer-reviewed on its merits – is the question important, is the design sound? – and, if accepted, receives in-principle acceptance. This means that publication is largely decoupled from the direction or significance of the eventual findings, provided the authors follow the approved protocol and meet the journal’s requirements at the final stage.
The effect on incentives is striking. One analysis found that 96% of studies in the standard psychology literature reported results supporting their hypothesis, compared with just 44% of Registered Reports. That difference highlights the extent to which the conventional publication process can favour positive findings – and how a results-blind review process may help counter that bias. Crucially, Registered Reports change what the publication decision rewards: researchers are evaluated primarily on the importance of the question and quality of the study design, rather than whether the eventual results are sufficiently novel, positive, or surprising.
It’s not a universal fix, though. While Registered Reports do target unclear incentives directly, since publication is no longer dependent on producing flashy results, the trade-off around being scooped is more complicated. A public, timestamped protocol can provide evidence that you were already pursuing an idea, but sharing a research plan before data collection may also increase the perceived risk of exposing it to competitors. In that sense, Registered Reports may help establish priority while potentially increasing idea-exposure risk.
But perhaps that fear is itself worth questioning. Is “scooping” not partly the product of a mis-incentivised system? In a system that rewarded reproducibility and cumulative evidence, a second study asking the same question wouldn’t necessarily be a threat—it could be exactly what strengthens confidence in the answer.
Finally, Registered Reports don’t necessarily solve the time limitation; instead, they shift more of the work upstream, requiring greater investment in study design, power calculations, analysis planning, and peer review before data collection. But there is an important distinction between saving an individual researcher’s time and saving the scientific community’s time. Better-designed, appropriately powered studies – and greater visibility of null and negative findings – give researchers more informative evidence to build upon and may prevent unnecessary duplication. Over the longer term, this could speed scientific progress: not because each study becomes quicker, but because the collective research effort becomes more informative and less wasteful.
Registered Reports therefore offer an interesting example of how seemingly cultural problems in science can be addressed by changing the structures researchers work within. They don’t eliminate the costs of rigorous and open science – and may sometimes redistribute them – but they demonstrate that incentives are not fixed: changing whenpublication decisions are made, and what those decisions reward, can change the evidence that enters the scientific literature.
They also remain far from universal. One study reported that, as of mid-2023, around 278 journals had adopted the format – a snapshot rather than a current total. Registered Reports are worth knowing about and reflecting on, even if it may be some time before they become a standard approach to scientific publishing.
By Teresa Simoes Steyn