3 ms·

> I am not sure you appreciate the scope of distrusting submissions. What you're saying is, the first option I gave, to fix the peer review process so it is a

by pdonis 3y ago

> I am not sure you appreciate the scope of distrusting submissions. What you're saying is, the first option I gave, to fix the peer review process so it is a good indicator of the reliability of research, is not feasible. If that's the case, that just means we have to pick the second option: stop treating peer reviewed research as if it is reliable.

beowulfey 3y ago
That's actually exactly correct. Peer review is supposed to characterize the research in terms of quality, not validate it. Otherwise, those peers who validate it would practically be co-authors. It's a system built upon honesty. A dishonest person is obviously going to take advantage of it. Really blatant things are caught in peer review, but sneaky fraud will definitely slip through.
- pdonis 3y ago
  > Peer review is supposed to characterize the research in terms of quality Fraudulent research would obviously not be of the desired quality, wouldn't it? > It's a system built upon honesty. But if the purpose is to characterize the research in terms of quality, this can't be right. Since fraud is known to happen, and there are known to be incentives for fraud, any process that claims to judge the quality of research would have to have some way of checking for fraud.
  - beowulfey 3y ago
    I think you're misunderstanding what I mean by "quality" in this context. It's not quality as in something like meat, it's quality as in "a quality education". Does it go above and beyond vs. do the bare minimum to address the scientific question proposed by the authors. Something can appear to be of very high quality and yet still be completely fraudulent. This occurs in countless fields; art, counterfeit money, deepfakes, etc. Scientists are attempting to address fraudulent research in other ways, but peer review is not the best means for it and it should not be thought of in that way.
    - pdonis 3y ago
      > Something can appear to be of very high quality and yet still be completely fraudulent. So in other words, peer review does not judge actual quality. It only judges apparent quality. Which is not what anyone actually wants to know.
kergonath 3y ago
> If that's the case, that just means we have to pick the second option: stop treating peer reviewed research as if it is reliable. It is the case, and that second option is exactly what we do. One article is nice, but replication is key before some new finding is accepted. That is exactly what happened with LK-99, for example, which is interesting because it was very public.
- pdonis 3y ago
  > that second option is exactly what we do I don't think so. I see claims made all the time about science being "settled" purely on the basis of it being peer reviewed. > replication is key before some new finding is accepted If only that were true. In many cases it isn't.

by pdonis 3y ago

beowulfey 3y ago

That's actually exactly correct. Peer review is supposed to characterize the research in terms of quality, not validate it. Otherwise, those peers who validate it would practically be co-authors. It's a system built upon honesty. A dishonest person is obviously going to take advantage of it. Really blatant things are caught in peer review, but sneaky fraud will definitely slip through.

pdonis 3y ago
> Peer review is supposed to characterize the research in terms of quality Fraudulent research would obviously not be of the desired quality, wouldn't it? > It's a system built upon honesty. But if the purpose is to characterize the research in terms of quality, this can't be right. Since fraud is known to happen, and there are known to be incentives for fraud, any process that claims to judge the quality of research would have to have some way of checking for fraud.
- beowulfey 3y ago
  I think you're misunderstanding what I mean by "quality" in this context. It's not quality as in something like meat, it's quality as in "a quality education". Does it go above and beyond vs. do the bare minimum to address the scientific question proposed by the authors. Something can appear to be of very high quality and yet still be completely fraudulent. This occurs in countless fields; art, counterfeit money, deepfakes, etc. Scientists are attempting to address fraudulent research in other ways, but peer review is not the best means for it and it should not be thought of in that way.
  - pdonis 3y ago
    > Something can appear to be of very high quality and yet still be completely fraudulent. So in other words, peer review does not judge actual quality. It only judges apparent quality. Which is not what anyone actually wants to know.

kergonath 3y ago

> If that's the case, that just means we have to pick the second option: stop treating peer reviewed research as if it is reliable. It is the case, and that second option is exactly what we do. One article is nice, but replication is key before some new finding is accepted. That is exactly what happened with LK-99, for example, which is interesting because it was very public.

pdonis 3y ago
> that second option is exactly what we do I don't think so. I see claims made all the time about science being "settled" purely on the basis of it being peer reviewed. > replication is key before some new finding is accepted If only that were true. In many cases it isn't.