3 ms·

Barriers to publication should be lower for replication studies, I think that’s the main problem. If someone wants to spend some time replicating something tha

Barriers to publication should be lower for replication studies, I think that’s the main problem. If someone wants to spend some time replicating something that’s only been described in a paper or two, that is valuable work for the community and should be encouraged. If the person is a PhD student using that as an opportunity to hone their skills, it’s even better. It’s not glamorous, it’s not something entirely new, but it is useful and important. And this work needs to go to normal journals, otherwise there’s just be journals dedicated to replication and their impact factor will be terrible and nobody will care.

s1artibartfast 3y ago
They're basically no barriers to publication. There are a number of normal journals that publish everything submitted if it appears to be honest research.
- kergonath 3y ago
  Not nice journals, though. At least not in my experience but that’s probably very field-dependent. It’s not uncommon to get a summary rejection letter for lack of novelty and that is one aspect they stress when they ask us to review articles.
  - s1artibartfast 3y ago
    But novelty IS what makes those journals nice and prestigious in the first place. It is the basis of their reputation. It's basically a catch 22. We want replication in prestigious journals, but any Journal with replications becomes less novel and prestigious. It all comes down to what people value about journals. If people valued replication more than novelty, replication journals would be the prestigious ones. It all comes back to the fact that doing novel science is considered more prestigious than replication. Institutions can play all kinds of games to try to make it harder for readers to tell novelty apart from replication, but people will just find new ways to signal and determine the difference. Let's say we pass a law that prestigious journals must published 50% replications. The Prestige from publishing in that journal will just shift to publishing in that journal with something like first demonstration in the title or publishing in that journal Plus having a high citation or impact value. It is really difficult to come up with the system or institution level solution when novelty is still what individuals value. As long as companies and universities value innovation, figure out ways to determine which scientists are innovative, and value them more
jxramos 3y ago
I wonder if undergrads could be harnessed to enter into this kind of work, maybe under the supervision of doctoral students and a well meaning and interested PI.

s1artibartfast 3y ago

They're basically no barriers to publication. There are a number of normal journals that publish everything submitted if it appears to be honest research.

kergonath 3y ago
Not nice journals, though. At least not in my experience but that’s probably very field-dependent. It’s not uncommon to get a summary rejection letter for lack of novelty and that is one aspect they stress when they ask us to review articles.
- s1artibartfast 3y ago
  But novelty IS what makes those journals nice and prestigious in the first place. It is the basis of their reputation. It's basically a catch 22. We want replication in prestigious journals, but any Journal with replications becomes less novel and prestigious. It all comes down to what people value about journals. If people valued replication more than novelty, replication journals would be the prestigious ones. It all comes back to the fact that doing novel science is considered more prestigious than replication. Institutions can play all kinds of games to try to make it harder for readers to tell novelty apart from replication, but people will just find new ways to signal and determine the difference. Let's say we pass a law that prestigious journals must published 50% replications. The Prestige from publishing in that journal will just shift to publishing in that journal with something like first demonstration in the title or publishing in that journal Plus having a high citation or impact value. It is really difficult to come up with the system or institution level solution when novelty is still what individuals value. As long as companies and universities value innovation, figure out ways to determine which scientists are innovative, and value them more

jxramos 3y ago

I wonder if undergrads could be harnessed to enter into this kind of work, maybe under the supervision of doctoral students and a well meaning and interested PI.