3 ms·
I don't agree with everything in this article, but just to give my 2 cents as somebody who has been on both sides of the equation: Before I ever did any review
by paraknight 5y ago
I don't agree with everything in this article, but just to give my 2 cents as somebody who has been on both sides of the equation:
Before I ever did any reviewing, I always felt like some conferences really are like a lottery on whether or not you get accepted. You can submit the same paper and get nothing but strong accepts or strong rejects maybe even with the same reviewers.
When reviewing papers, I think I understood more about why. When reviewing you can see other reviewers' comments when you're done with your review, as well as their level of expertise. With some really really niche areas, some people simply aren't qualified to be reviewing, specify their expertise in an area as higher than it is, and give a really half-assed review. Part of that is also because we need to review a whole bunch of papers and can't dedicate hours to each one.
The only time I've found this process to work properly is with really good papers or really shit papers. There, everybody tends to agree. For average papers, it's not that you have a mixture of weak accepts and weak rejects and it comes down to the wire, but rather it's a mixture of strong accepts and strong rejects, with comments that praise/roast a paper that you're not even sure is the same one you read, and you doubt your own understanding of the topic. There is usually very little structure and objectivity built into the system, so mediocre science presented really well is much more likely to do well. That's why it's so random.