4 ms·

Peer review is pretty unpopular round these parts. In mathematics/TCS I've had mostly good experiences. Actually most of the time the review process improved my

Peer review is pretty unpopular round these parts. In mathematics/TCS I've had mostly good experiences. Actually most of the time the review process improved my papers. Clearly something is rotten about the way peer review is implemented in the empirical sciences. But think of all those high profile retractions you read about these days. Usually that comes about by a sort of post hoc peer review, not by anything resembling market forces.

YeGoblynQueenne 3y ago
Not to be mean to the HN community but at least a substantial minority of people who complain about peer review on here have no experience of peer review, even in applied CS and AI and machine learning, which are the hot topics today. But they've read that peer review is broken and, by god, they'll let the world know! For science! I publish in machine learning and my experience is the same as yours: reviews have mainly helped me to improve my papers. Though to be fair this is mainly the case in journals; in conferences it's true that reviewers will often look for reasons to reject and don't try to be constructive (I always do; I still find it very hard to reject). This is the result of the field of AI research having experienced a huge explosion of interest, and therefore submissions, in the last few years, so that all the conferences are creaking under the strain. Most of the new entrants are also junior researchers without too much experience- and that is true for both authors and reviewers (who are only invited to review after they publish in a venue). So the conferences are a bit of a mess at the moment, and the quality of the papers that get submitted and published, overall low. But that's not because "peer review is broken", it's because too many people start a PhD in machine learning thinking they'll get immediately hired by Google, or OpenAI I guess. That too shall pass, and then things will calm down.
- ccppurcell 3y ago
  Agreed wholeheartedly but didn't want to come out swinging! Not only that but my experience of reviewing has also been positive, and has given me ideas for research and how to present research (notation, paper structure etc) Except my first review which was a 100 page survey paper on a very specific kind of inequality that exists for practically any graph invariant, so every page was pretty much identical just with alpha then beta then omega then chi... And the deadline was my birthday!

YeGoblynQueenne 3y ago

Not to be mean to the HN community but at least a substantial minority of people who complain about peer review on here have no experience of peer review, even in applied CS and AI and machine learning, which are the hot topics today. But they've read that peer review is broken and, by god, they'll let the world know! For science! I publish in machine learning and my experience is the same as yours: reviews have mainly helped me to improve my papers. Though to be fair this is mainly the case in journals; in conferences it's true that reviewers will often look for reasons to reject and don't try to be constructive (I always do; I still find it very hard to reject). This is the result of the field of AI research having experienced a huge explosion of interest, and therefore submissions, in the last few years, so that all the conferences are creaking under the strain. Most of the new entrants are also junior researchers without too much experience- and that is true for both authors and reviewers (who are only invited to review after they publish in a venue). So the conferences are a bit of a mess at the moment, and the quality of the papers that get submitted and published, overall low. But that's not because "peer review is broken", it's because too many people start a PhD in machine learning thinking they'll get immediately hired by Google, or OpenAI I guess. That too shall pass, and then things will calm down.

ccppurcell 3y ago
Agreed wholeheartedly but didn't want to come out swinging! Not only that but my experience of reviewing has also been positive, and has given me ideas for research and how to present research (notation, paper structure etc) Except my first review which was a 100 page survey paper on a very specific kind of inequality that exists for practically any graph invariant, so every page was pretty much identical just with alpha then beta then omega then chi... And the deadline was my birthday!