3 ms·

Not in academia, but what I hear is that very few results are ever attempted to be reproduced. So if you publish an unreproducible paper, you can probably have

Not in academia, but what I hear is that very few results are ever attempted to be reproduced. So if you publish an unreproducible paper, you can probably have a full career without anyone noticing.

jfengel 2y ago
Papers that can't be reproduced sound like they're not very useful, either. I know it's not as simple as that, and "useful" can simply mean "cited" (a sadly overrated metric). But surely it's easier to get hired if your work actually results in something somebody uses.
- dgfitz 2y ago
  > Papers that can't be reproduced sound like they're not very useful, either. They’re not useful at all. Reproduction of results isn’t sexy, nobody does it. Almost feels like science is built on a web on funding trying to buy the desired results.
  - qpiox 2y ago
    Reproduction is rarely done because it is not "new science". Everyone is funding only "new science".
  - jfengel 2y ago
    Reproduction is boring, but it would often happen incidentally to building off someone else's results. You tell me that this reaction creates X, and I need X to make Y. If I can't make my Y, sooner or later it's going to occur to me that X is the cause. Like I said, I know it's never that easy. Bench work is hard and there are a million reasons why your idea failed, and you may not take the time to figure out why. You won't report such failures. And complicated results, like in sociology, are rarely attributable to anything.
    - mike_hearn 2y ago
      That's true for some kinds of research but a lot of academic output isn't as firm as "X creates Y". Replicability is overrated anyway. Loads of bad papers will replicate just fine if you try. They're still making false claims. https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280 https://blog.plan99.net/replication-studies-cant-fix-science...
  - air7 2y ago
    I've had this idea that reproduction studies in one's C.V should become a sort of virtue signal, akin to philanthropy among the rich. This way, some percentage of one's work would need to be reproduction work or otherwise they would be looked down upon, and this would create the right incentive to do go.
- qpiox 2y ago
  The reality is a bit different. The "better" journals are listed in JCR. Nearly 40% of them have impact factor less than 1, it means that on average papers in them are cited less than 1 times. Conclusion: even in better journals, the average paper is rarely cited at all, which means that definitely the public has rarely heard of it or found it useful.
- gopher_space 2y ago
  Papers are reproducible in exactly the same way that github projects are buildable, and in both cases anything that comes fully assembled for you is already a product. If your academic research results in immediately useful output all of the people waiting for that to happen step in and you no longer worry about employment.
77pt77 2y ago
Depends on the field. Psycho* is rife with that.
- belter 2y ago
  > Depends on the field. Yeah...It's more on the less Pure domains...And mostly overseas?... :-) https://xkcd.com/435/ https://xkcd.com/435/ "A 2016 survey by Nature on 1,576 researchers who took a brief online questionnaire on reproducibility found that more than 70% of researchers have tried and failed to reproduce another scientist's experiment results (including 87% of chemists, 77% of biologists, 69% of physicists and engineers, 67% of medical researchers, 64% of earth and environmental scientists, and 62% of all others), and more than half have failed to reproduce their own experiments." https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
  - 77pt77 2y ago
    > And mostly overseas Where is the data that supports that? And by overseas what do you mean, or are we talking about USA "defaultism" here?

Not in academia, but what I hear is that very few results are ever attempted to be reproduced. So if you publish an unreproducible paper, you can probably have a full career without anyone noticing.

jfengel 2y ago

Papers that can't be reproduced sound like they're not very useful, either. I know it's not as simple as that, and "useful" can simply mean "cited" (a sadly overrated metric). But surely it's easier to get hired if your work actually results in something somebody uses.

dgfitz 2y ago
> Papers that can't be reproduced sound like they're not very useful, either. They’re not useful at all. Reproduction of results isn’t sexy, nobody does it. Almost feels like science is built on a web on funding trying to buy the desired results.
- qpiox 2y ago
  Reproduction is rarely done because it is not "new science". Everyone is funding only "new science".
- jfengel 2y ago
  Reproduction is boring, but it would often happen incidentally to building off someone else's results. You tell me that this reaction creates X, and I need X to make Y. If I can't make my Y, sooner or later it's going to occur to me that X is the cause. Like I said, I know it's never that easy. Bench work is hard and there are a million reasons why your idea failed, and you may not take the time to figure out why. You won't report such failures. And complicated results, like in sociology, are rarely attributable to anything.
  - mike_hearn 2y ago
    That's true for some kinds of research but a lot of academic output isn't as firm as "X creates Y". Replicability is overrated anyway. Loads of bad papers will replicate just fine if you try. They're still making false claims. https://blog.plan99.net/replication-studies-cant-fix-science-0e195234a280 https://blog.plan99.net/replication-studies-cant-fix-science...
- air7 2y ago
  I've had this idea that reproduction studies in one's C.V should become a sort of virtue signal, akin to philanthropy among the rich. This way, some percentage of one's work would need to be reproduction work or otherwise they would be looked down upon, and this would create the right incentive to do go.
qpiox 2y ago
The reality is a bit different. The "better" journals are listed in JCR. Nearly 40% of them have impact factor less than 1, it means that on average papers in them are cited less than 1 times. Conclusion: even in better journals, the average paper is rarely cited at all, which means that definitely the public has rarely heard of it or found it useful.
gopher_space 2y ago
Papers are reproducible in exactly the same way that github projects are buildable, and in both cases anything that comes fully assembled for you is already a product. If your academic research results in immediately useful output all of the people waiting for that to happen step in and you no longer worry about employment.

77pt77 2y ago

Depends on the field. Psycho* is rife with that.

belter 2y ago
> Depends on the field. Yeah...It's more on the less Pure domains...And mostly overseas?... :-) https://xkcd.com/435/ https://xkcd.com/435/ "A 2016 survey by Nature on 1,576 researchers who took a brief online questionnaire on reproducibility found that more than 70% of researchers have tried and failed to reproduce another scientist's experiment results (including 87% of chemists, 77% of biologists, 69% of physicists and engineers, 67% of medical researchers, 64% of earth and environmental scientists, and 62% of all others), and more than half have failed to reproduce their own experiments." https://en.wikipedia.org/wiki/Replication_crisis https://en.wikipedia.org/wiki/Replication_crisis
- 77pt77 2y ago
  > And mostly overseas Where is the data that supports that? And by overseas what do you mean, or are we talking about USA "defaultism" here?