4 ms·
I think this is not even really a problem with the peer review system, but a problem with the (mostly, nonexistence of) an in depth science journalism field. Pe
by bee_rider 3y ago
I think this is not even really a problem with the peer review system, but a problem with the (mostly, nonexistence of) an in depth science journalism field. Peer review just means it is ok to stick in a scientific journal and have peers read it… people with advanced bullshit detectors and nuance parsers in a particular domain.
A paper is a brick. A policy is a building. It is not a problem in the brick manufacturing field, if people keep trying to build houses without any mortar.
(Just to be explicit, I think you are right on the money and just wanted to elaborate/rant).
- thaumasiotes 3y ago> Peer review just means it is ok to stick in a scientific journal and have peers read it… people with advanced bullshit detectors and nuance parsers in a particular domain. That's not what peer review means. If that were what it meant, it would have no purpose - the people you describe don't benefit from the paper they're reading having been through peer review. They already understand the domain. Compare https://genomesunzipped.org/2012/08/the-first-steps-towards-a-modern-system-of-scientific-publication.php https://genomesunzipped.org/2012/08/the-first-steps-towards-... : > The solution to this problem relies on a simple observation – in my field, I am completely indifferent to whether a paper has been “peer-reviewed” for the basic reason that I consider myself a “peer”. I do not think it extremely hubristic to say that I am reasonably capable of evaluating whether a paper in my field is worth reading, and then if so, of judging its merits. The opinions of other people in the field are of course important, but in no way does the fact that two or three nameless people thought a paper worth publishing influence my opinion of it. The only goal that peer review serves is letting people who aren't peers and don't understand what the paper is saying believe that they can tell whether the paper is good or not. They still can't, but because of the peer review system, they feel that they can.
- bee_rider 3y agoIt isn’t pointless, researchers have finite time and attention, the point of peer review is to keep out the really bad papers so that they don’t waste their time on them.
- godelski 3y ago> I think this is not even really a problem with the peer review system, but a problem with the (mostly, nonexistence of) an in depth science journalism field. Peer review just means it is ok to stick in a scientific journal and have peers read it… people with advanced bullshit detectors and nuance parsers in a particular domain. I don't agree with this. There is deception in its name: peer review. Even "preprints" (things on arxiv and not accepted in journals or conferences) are still read and evaluated by peers. LK-99 is a great example of this, and that's not an uncommon process. The problem is that venues perpetuate the idea that being published within them is a mark of validity. But, like you suggest, it isn't. And "peer review" doesn't mean that it is just okay for peers to read it because we're reading each others works long before they are released in venues. Unfortunately most people (including researchers) don't have advanced bullshit detectors or nuance parsers. These are often fairly narrow and there's not that strong of alignment between the subject matter that a reviewer is reviewing and their niche domain expertise. I agree journalism and politicians need to do better, but so do we in the scientific community. There's more nuance. It is definitely a problem with the brick manufacturer if the marketing team implies you don't need mortar or even says "generally you don't need mortar."