They looked at studies in top psychology, economics, and nature / science journals and found that only 39% of 100 psychology articles were successfully reprinted. Replication rates were 61% for 18 economic studies and 62% for 21 nature / science studies.
But papers whose results could not be reproduced later attracted a lot of attention at the time of publication: they were cited 153 times more often than those whose conclusions could be repeated.
The biggest discrepancy was in nature / science articles, where those that could not be reproduced were cited 300 times more often than reproducible articles, according to the study from the University of California at San Diego.
Unverifiable research tends to be cited as if the results were true long after the post failed to replicate, the researchers noted.
“We also know that experts can predict well which papers will be reproduced,” wrote authors Marta Serra-Garcia, assistant professor of economics and strategy, and Uri Gneezy, professor of behavioral economics. “Given this prediction, we ask ourselves, ‘Why are non-reproducible papers accepted for publication in the first place?'”
One possible reason is that journals use less stringent replication standards if a study is more interesting, the researchers suggested.
“Interesting or appealing findings are also getting more media coverage or shared on platforms like Twitter, which gets a lot of attention, but it doesn’t make them true,” Gneezy said in a college press release.
The publication of unverified studies can have significant and lasting impacts, according to the study’s authors. They pointed to a bogus study published in The Lancet in 1998 that implicated a link between the measles, mumps, and rubella vaccine and autism, which has led tens of thousands of parents around the world to turn against it. vaccine.
The incorrect results were retracted by The Lancet 12 years later, but many people still cite the study to claim that autism is linked to vaccines.
“We hope our research encourages readers to be careful if they read something interesting and engaging,” Serra-Garcia said in the release.
“Whenever researchers cite work that is more interesting or has been cited a lot, we hope they will check to see if replication data is available and what those results suggest,” Gneezy said.