3 ms·

I don't think the process would be less effective if papers were published without peer review, but were then questioned and challenged in public debate under e

I don't think the process would be less effective if papers were published without peer review, but were then questioned and challenged in public debate under editorial moderation. Peer review isn't a single-goal process. It exists to introduce citations of existing work if they aren't already referenced, to question assumptions, and to find obvious errors. (Among others.) All of these could be done in peer debate. It's not as if the process is genuinely anonymous anyway. Of course it pretends to be, but it isn't. Peer review dates from the days when information travelled as fast as a horse, and it made sense to include a review stage for basic quality control. I'm not convinced it's a good model now, when information exchange is rather quicker. The real problem is that journal publication is about career status, not about quality of work. Peer review is almost a side issue. The scientific publishing industry is not just financially parasitic, it also introduces unnecessary delays in idea exchange. Public peer debate would eliminate some of that friction, but it would also have public consequences for reputation - and that might well be more effective for quality control than pseudo-anonymity.

michaelmior 6y ago
At least in my field, I expect this would be a total disaster. One of the major advantages of peer review is that knowing the top tier publication venues, I can look to these to keep up to date on the latest work knowing that what I find is likely to be high quality. In the absence of peer-reviewed publication venues, I see two options: 1. Read everything I can find that is published and try to sort through it all myself to sort out what is good quality. I think in aggregate, this is likely to take more of researchers' time than contributing to the peer review process. 2. Focus only on publications from top quality institutions or researchers whose work I am already familiar with. This has the advantage of saving me time. But it also has the massive disadvantage that I am likely to miss quality work which comes from lesser known institutions or authors. I don't think either of those two options is attractive. In my field, the delay of idea exchange is not really a problem. If people want to share their ideas, they can often publish a preprint in parallel with submission to a peer-reviewed journal. (Although some journals effectively prohibit this since it has the potential to break the anonymity of the review process.) Yes, some of the purposes of peer review could be accomplished in the method of "peer debate" you mention. But I think such a process would be much less effective. Of course, that's not to say I don't think there are things broken about peer review that could use fixing :)

michaelmior 6y ago

At least in my field, I expect this would be a total disaster. One of the major advantages of peer review is that knowing the top tier publication venues, I can look to these to keep up to date on the latest work knowing that what I find is likely to be high quality. In the absence of peer-reviewed publication venues, I see two options: 1. Read everything I can find that is published and try to sort through it all myself to sort out what is good quality. I think in aggregate, this is likely to take more of researchers' time than contributing to the peer review process. 2. Focus only on publications from top quality institutions or researchers whose work I am already familiar with. This has the advantage of saving me time. But it also has the massive disadvantage that I am likely to miss quality work which comes from lesser known institutions or authors. I don't think either of those two options is attractive. In my field, the delay of idea exchange is not really a problem. If people want to share their ideas, they can often publish a preprint in parallel with submission to a peer-reviewed journal. (Although some journals effectively prohibit this since it has the potential to break the anonymity of the review process.) Yes, some of the purposes of peer review could be accomplished in the method of "peer debate" you mention. But I think such a process would be much less effective. Of course, that's not to say I don't think there are things broken about peer review that could use fixing :)