4 ms·

Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it. It doesn't. Not even close. Peer review does

by MarkusQ 8mo ago

Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it. It doesn't. Not even close. Peer review doesn't even mean that it's free from errors, free from fraud, free from methodological mischief; it doesn't mean anything at this point. Yet we continue to act like it does. Darwin's work wasn't peer reviewed. Nor Einstein's. It's something we cooked up in the mid 1900's to deal with the fallout from another mistake ("publish or perish") that meant people had to try to publish even if they had nothing to say.

bonsai_spool 8mo ago
> Darwin's work wasn't peer reviewed. Nor Einstein's Except it was…? This is absurdly ahistorical and the fact that you cross disciplines in trying to make an incorrect argument questions whether you are in science at all. The structure of peer review in Darwin’s time was different, where experts wrote monographs and gave lectures at symposia that then led to letters among their peers. Which is what happens now, if you take a step back. The volume of new work these days is incompatible with the older informal system, and is in some ways our new paradigm is superior as there is a formal period in which new works are reviewed.
- MarkusQ 8mo ago
  Sorry. I meant "peer reviewed prior to publication" as the phrase is presently used. I thought that was obvious. What you're calling "peer review" is what I would call "discussed" or "debated" which it certainly was. I dispute your claim that the new paradigm is superior.
  - bonsai_spool 8mo ago
    > Sorry. I meant "peer reviewed prior to publication" as the phrase is presently used. Accepted. But now there is Arxiv and Biorxiv and even Medrxiv—so we're back to where things were, it seems.
snowwrestler 8mo ago
> Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it. No actual working scientist thinks this. “Glitchc” has it right elsewhere in this thread: the motivating force behind journals is prominence and reputation, not truth.
- D-Machine 8mo ago
  Ah, but the naive public still broadly believes in peer review, and that high profile journals do good review. And the prominence and reputation that comes from these journals arguably then relies on this (increasingly false) public perception. Would scientists feel the same if the public was more educated about how bad journals and peer review are? Not so easy to disentangle IMO.
  - snowwrestler 8mo ago
    The naive public does not believe anything in particular about peer review. They think new scientific results are significant when they read about them in the popular media, that’s it. People who do need to work professionally with peer review, do understand what it actually does and its limitations. You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand.
    - D-Machine 8mo ago
      > The naive public does not believe anything in particular about peer review You'd need to provide evidence or an argument for this. The media reports on things in part based on journal prestige, and likely when questioned, people will say they can trust such things because good scientists have looked at the work and say it is good. This would be an implicit belief that peer review is generally working well, even if they don't use the term "peer review". > You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand. Extremely presumptuous, as I work in this system, and have provided plenty of evidence for my claims. You've provided only sneers.
D-Machine 8mo ago
Don't know why you are being downvoted, you are largely correct. I've provided plenty of evidence in another post in this thread showing that journal-based peer review is highly farcical. EDIT: I still want review from a community of scientific peers. I just don't want this review to be in the hands of a tiny number of gatekeepers entangled with journals that largely just slow things down.
- MarkusQ 8mo ago
  Because a lot of people are deeply invested in the present system perhaps? As the article pointed out, there's a lot of money involved, and there are a lot of people who've built their lives around flourishing in the existing system, cut-throat as it may be.
  - D-Machine 8mo ago
    > Because a lot of people are deeply invested in the present system perhaps? I mean, right, yes, of course. Much of the downvotes are cognitive dissonance, obviously. I suppose I meant the question rhetorically.

by MarkusQ 8mo ago

bonsai_spool 8mo ago

> Darwin's work wasn't peer reviewed. Nor Einstein's Except it was…? This is absurdly ahistorical and the fact that you cross disciplines in trying to make an incorrect argument questions whether you are in science at all. The structure of peer review in Darwin’s time was different, where experts wrote monographs and gave lectures at symposia that then led to letters among their peers. Which is what happens now, if you take a step back. The volume of new work these days is incompatible with the older informal system, and is in some ways our new paradigm is superior as there is a formal period in which new works are reviewed.

MarkusQ 8mo ago
Sorry. I meant "peer reviewed prior to publication" as the phrase is presently used. I thought that was obvious. What you're calling "peer review" is what I would call "discussed" or "debated" which it certainly was. I dispute your claim that the new paradigm is superior.
- bonsai_spool 8mo ago
  > Sorry. I meant "peer reviewed prior to publication" as the phrase is presently used. Accepted. But now there is Arxiv and Biorxiv and even Medrxiv—so we're back to where things were, it seems.

snowwrestler 8mo ago

> Part of the problem is we got tricked into thinking "peer reviewed" meant "true," or at least something like it. No actual working scientist thinks this. “Glitchc” has it right elsewhere in this thread: the motivating force behind journals is prominence and reputation, not truth.

D-Machine 8mo ago
Ah, but the naive public still broadly believes in peer review, and that high profile journals do good review. And the prominence and reputation that comes from these journals arguably then relies on this (increasingly false) public perception. Would scientists feel the same if the public was more educated about how bad journals and peer review are? Not so easy to disentangle IMO.
- snowwrestler 8mo ago
  The naive public does not believe anything in particular about peer review. They think new scientific results are significant when they read about them in the popular media, that’s it. People who do need to work professionally with peer review, do understand what it actually does and its limitations. You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand.
  - D-Machine 8mo ago
    > The naive public does not believe anything in particular about peer review You'd need to provide evidence or an argument for this. The media reports on things in part based on journal prestige, and likely when questioned, people will say they can trust such things because good scientists have looked at the work and say it is good. This would be an implicit belief that peer review is generally working well, even if they don't use the term "peer review". > You seem stuck somewhere in the middle, caring deeply about a system you don’t seem to fully understand. Extremely presumptuous, as I work in this system, and have provided plenty of evidence for my claims. You've provided only sneers.

D-Machine 8mo ago

Don't know why you are being downvoted, you are largely correct. I've provided plenty of evidence in another post in this thread showing that journal-based peer review is highly farcical. EDIT: I still want review from a community of scientific peers. I just don't want this review to be in the hands of a tiny number of gatekeepers entangled with journals that largely just slow things down.

MarkusQ 8mo ago
Because a lot of people are deeply invested in the present system perhaps? As the article pointed out, there's a lot of money involved, and there are a lot of people who've built their lives around flourishing in the existing system, cut-throat as it may be.
- D-Machine 8mo ago
  > Because a lot of people are deeply invested in the present system perhaps? I mean, right, yes, of course. Much of the downvotes are cognitive dissonance, obviously. I suppose I meant the question rhetorically.