4 ms·

You actually touch on an interesting point. Is peer review still necessary? When there is limited journal space, sure. But now that we have effectively unlimite

You actually touch on an interesting point. Is peer review still necessary? When there is limited journal space, sure. But now that we have effectively unlimited space, let it be reviewed when people go to cite it. We've already seen a lot of peer reviewed papers be retracted later, so should we just accept the reality?

dazed_confused 3y ago
Yes. The crap people publish even with peer-reviews is bad enough. A few subfields in CS have bitten a couple times by relying on non peer reviewed ecosystems.
gwerbret 3y ago
Peer review is the backbone of journals, and (some) journals serve an absolutely critical function in academia: they increase the signal:noise ratio. Most of what is published is noise; without the curation provided by (certain) journals, anything of significance is very likely to be drowned out. As a casual example in the biomedical sciences, the Journal of Biological Chemistry has an output of ~30,000 pages per year, most of which is 'noise'. That's just ONE journal. The journal Cell, on the other hand, has an order of magnitude less, most of which is 'signal'. EDIT: This is not to say the peer review approach doesn't need work, and lots of it. The whole current approach to research needs an overhaul. I'm just saying it's a bit hasty to throw the baby out with the bath water.
freedomben 3y ago
Peer review is an important signal to potential citers. If everyone has to fully read and understand every paper in order to (responsibly) cite it, there's gonna be a lot less research. Given the exposure of how much bad research there is, maybe there does need to be a slowing and focus on quality, but I think we still need peer review, although it definitely need to be reformed somehow because it's clearly broken. We need to take a serious look at the incentive structure in academia because it's not guaranteeing the scientific results that we expected it to. I don't think we should just abandon the system though.
- llm_trw 3y ago
  Peer review can be very simply replaced by skimming the abstract. If the abstract doesn't make sense the paper doesn't make sense either. All the disciplines which use arxiv as their main journal are doing well enough without peer review.
adastra22 3y ago
No, it is not necessary. Just as it was not necessary for the first 300 years of the Royal society’s existence. “Peer review” used to be just a stand in phrase for the marketplace of ideas—seeing how your peers evaluate your work in a public process of getting published and sending letters to the editor. Only in living memory has it been warped into this pre-publication gatekeeping process.
bumby 3y ago
Necessary to share ideas? No. Necessary to share important ideas? Maybe. One of the aspects of journals that I don't see talked about much here is the curation of articles. Saying (good) journals aren't necessary is like saying journalists aren't necessary to get your news in the internet era. Not all information is necessarily good, and we often rely on systems/people to help winnow the amount we have to sift through. This is important in a society where the sheer amount of information developed far outpaces what we can consume in several lifetimes.
- llm_trw 3y ago
  It's more like saying that maybe we don't need the NYT, the people who lied about Iraq: https://www.theguardian.com/world/2004/may/27/media.iraq https://www.theguardian.com/world/2004/may/27/media.iraq, to get good quality journalism when the internet exists. Journals are in the same boat. Any good they have done is dwarfed by the few 'honest mistakes', like Alzheimer's cabal: https://www.statnews.com/2019/06/25/alzheimers-cabal-thwarted-progress-toward-cure/ https://www.statnews.com/2019/06/25/alzheimers-cabal-thwarte..., which have done more to stifle science than anyone since Stalin picking which biological theories are socialist enough to be true.
  - bumby 3y ago
    I think you're conflating the notion of perfect curation. I don't think anyone is claiming that journals or journalists are infallible. They certainly make m, although I don't think I agree that the good is "dwarfed" by them, except when used as a direct attempt to undermine credibility by adversaries. Put differently, do you think bad ideas spread more easily with the internet? The current research seems to think so, and if you agree, I don't see how that mitigates the spread of bad information. If anything, it exacerbates it.
    - llm_trw 3y ago
      I think that every bad idea spread by the internet dwarfs the harms done by ideas spread by government. Every issue with the middle east over the last 20 years is the direct result of a crusade launched on a lie enthusiastically repeated by the same people claiming the moral high ground today.

dazed_confused 3y ago

Yes. The crap people publish even with peer-reviews is bad enough. A few subfields in CS have bitten a couple times by relying on non peer reviewed ecosystems.

gwerbret 3y ago

Peer review is the backbone of journals, and (some) journals serve an absolutely critical function in academia: they increase the signal:noise ratio. Most of what is published is noise; without the curation provided by (certain) journals, anything of significance is very likely to be drowned out. As a casual example in the biomedical sciences, the Journal of Biological Chemistry has an output of ~30,000 pages per year, most of which is 'noise'. That's just ONE journal. The journal Cell, on the other hand, has an order of magnitude less, most of which is 'signal'. EDIT: This is not to say the peer review approach doesn't need work, and lots of it. The whole current approach to research needs an overhaul. I'm just saying it's a bit hasty to throw the baby out with the bath water.

freedomben 3y ago

Peer review is an important signal to potential citers. If everyone has to fully read and understand every paper in order to (responsibly) cite it, there's gonna be a lot less research. Given the exposure of how much bad research there is, maybe there does need to be a slowing and focus on quality, but I think we still need peer review, although it definitely need to be reformed somehow because it's clearly broken. We need to take a serious look at the incentive structure in academia because it's not guaranteeing the scientific results that we expected it to. I don't think we should just abandon the system though.

llm_trw 3y ago
Peer review can be very simply replaced by skimming the abstract. If the abstract doesn't make sense the paper doesn't make sense either. All the disciplines which use arxiv as their main journal are doing well enough without peer review.

adastra22 3y ago

No, it is not necessary. Just as it was not necessary for the first 300 years of the Royal society’s existence. “Peer review” used to be just a stand in phrase for the marketplace of ideas—seeing how your peers evaluate your work in a public process of getting published and sending letters to the editor. Only in living memory has it been warped into this pre-publication gatekeeping process.

bumby 3y ago

Necessary to share ideas? No. Necessary to share important ideas? Maybe. One of the aspects of journals that I don't see talked about much here is the curation of articles. Saying (good) journals aren't necessary is like saying journalists aren't necessary to get your news in the internet era. Not all information is necessarily good, and we often rely on systems/people to help winnow the amount we have to sift through. This is important in a society where the sheer amount of information developed far outpaces what we can consume in several lifetimes.

llm_trw 3y ago
It's more like saying that maybe we don't need the NYT, the people who lied about Iraq: https://www.theguardian.com/world/2004/may/27/media.iraq https://www.theguardian.com/world/2004/may/27/media.iraq, to get good quality journalism when the internet exists. Journals are in the same boat. Any good they have done is dwarfed by the few 'honest mistakes', like Alzheimer's cabal: https://www.statnews.com/2019/06/25/alzheimers-cabal-thwarted-progress-toward-cure/ https://www.statnews.com/2019/06/25/alzheimers-cabal-thwarte..., which have done more to stifle science than anyone since Stalin picking which biological theories are socialist enough to be true.
- bumby 3y ago
  I think you're conflating the notion of perfect curation. I don't think anyone is claiming that journals or journalists are infallible. They certainly make m, although I don't think I agree that the good is "dwarfed" by them, except when used as a direct attempt to undermine credibility by adversaries. Put differently, do you think bad ideas spread more easily with the internet? The current research seems to think so, and if you agree, I don't see how that mitigates the spread of bad information. If anything, it exacerbates it.
  - llm_trw 3y ago
    I think that every bad idea spread by the internet dwarfs the harms done by ideas spread by government. Every issue with the middle east over the last 20 years is the direct result of a crusade launched on a lie enthusiastically repeated by the same people claiming the moral high ground today.