Mathematical modeler and statistics. Credit: Image is provided by the Anthro Illustrated project (https://anthroillustrated.com)

Current reforms to end the rat race between scientists can help; but are they enough?

Science is societys best method for understanding the world. Yet many scientists are unhappy with the way it works, and there are growing concerns that there is something broken in current scientific practice.

Many of the rules and procedures that are meant to promote innovative research are little more than historical precedents with little reason to suppose they encourage efficient or reliable discoveries. Worse, they can have perverse side-effects that harm both science and scientists. A well-known example is the general preference for positive over negative results, which creates a publication bias giving the false impression that certain effects exist, where in reality the dissenting evidence simply fails to be released.

Arizona State University researchers Thomas Morgan and Minhua Yan, working with ASU graduate Leonid Tiokhin, now at University of Technology Eindhoven in the Netherlands, have developed a new model, published this week in Nature Human Behaviour, to better understand the challenges facing the scientific process and how we can make it better. They focused on the priority rule: the tendency for the first scientist to document a finding to be disproportionately rewarded with prestige, prizes and career opportunities while those in second place get little to no recognition.

Many scientists have sleepless nights worrying about being scooped fearing that their work wont be considered novel enough for the highest-impact scientific journals because a different group working on the same topic manages to publish first. The priority rule has been around for centuries. In the 17th century, Isaac Newton and Gottfried Leibniz haggled over who invented calculus. And in the 19th century, Charles Darwin rushed to publish On the Origin of Species to avoid being scooped by Alfred Russel Wallace.

Rewarding priority is understandable and has some benefits. However, it comes at a cost, Tiokhin said. Rewards for priority may tempt scientists to sacrifice the quality of their research and cut corners.

The idea is that competition encourages scientists to work hard and efficiently, such that discoveries are made quickly, said Morgan, a research affiliate with theInstitute of Human Originsand associate professor with theSchool of Human Evolution and Social Change.But if everyone is working hard, and you need to come in first to be successful, then theres a temptation to cut corners to maximize your chances, even if it means the science suffers.

This is partly why some academic publishers, such as PLOS and eLife, now offer scoop protection, allowing researchers to publish findings identical to those already published within a certain timeframe. The problem is that science and publishers currently dont have a good idea about whether these reforms make sense.

To figure out how exactly the preference for priority affects science, and whether recent reforms offer any solution for its potential drawbacks, the collaborators developed an evolutionary agent-based model. This computer model simulates how a group of scientists investigate or abandon research questions, depending on their own results and the behavior of other scientists they compete against.The benefit of an evolutionary simulation is that we dont need to specify in advance how scientists behave. We just create a world in which success is rewarded, and we let selection figure out what kinds of behavior this favors, Morgan said. We can then vary what it means to successful for instance, whether or not its critical to come first and see how selection changes the behavior of scientists in response. We can also measure the benefit to society are scientists being efficient? Are their findings accurate? And so on.

The researchers found that a culture of excessive rewards for priority can have harmful effects. Among other things, it motivates scientists to conduct quick and dirty studies, so that they can be first to publish. This reduces the quality of their work and harms the reliability of science as a whole.

The model also suggests that scoop protection, as introduced by PLOS and eLife, works.

It reduces the temptation to rush the research and gives researchers more time to collect additional data, Tiokhin said. However, scoop protection is no panacea.

This is because scoop protection motivates some scientists to continue with a research line even after several results on that topic have been published, which reduces the total number of research questions the scientific community can address.

Scoop protection reforms in themselves, while helpful, are not sufficient to guarantee high-quality research or a reliable published literature. The model also shows that even with scoop protection, scientists will be tempted to run many small studies if new studies are cheap and easy to set up and the rewards for negative results are high. This suggests that measures that force scientists to invest more heavily in each study, such as asking scientists to preregister their studies or get their research plans criticized before they begin collecting data, can help.

We also learned that inefficiency in science is not always a bad thing. On the contrary inefficiencies force researchers to think twice before starting a new study, Tiokhin said.

Another option is to make large-scale data collection so straightforward that there is less incentive to skimp on data, alternatively, reviewers and journals could be more vigilant in looking out for underpowered studies with small sample sizes.

This project is an example of metascience, the use of the scientific method to study science itself.

It was a great pleasure to be part of this project. I got to use my modeling skills not only to make specific scientific discoveries, but also to shed light on how the scientific procedure itself should be designed to increase research quality and credibility. This benefits the whole scientific community and ultimately, the whole society, said Yan, a graduate student in the School of Human Evolution and Social Change.

Reference: Competition for priority harms the reliability of science, but reforms can help by Leonid Tiokhin, Minhua Yan and Thomas J. H. Morgan, 28 January 2021, Nature Human Behaviour.DOI: 10.1038/s41562-020-01040-1

Written by Julie Russ (ASU) and H.G.P van Appeven (Eindhoven University of Technology).

Link:
Ending the Rat Race: How Evolution Can Change Science for the Better - SciTechDaily