3 ms·

In general I would normally agree with this sentiment, but they're only looking at top tier conference venues. It's very difficult to pass lackluster work throu

by hayesbh 12y ago

In general I would normally agree with this sentiment, but they're only looking at top tier conference venues. It's very difficult to pass lackluster work through these places, so it really isn't that terrible of a metric. I would be more interested to see what would happen if they only included double-blind conferences, as name recognition in peer review can help make the rich richer. As another commenter said, there are few reliable signals that one could really use for this in general, let alone to get a current ranking. Citations are a fine metric but typically it will take a few years for those to really build up on a paper. I would be very interested to see them expand these rankings beyond theoretical CS and into the other disciplines... particularly robotics, where US News doesn't offer a ranking.

mjn 12y ago
> It's very difficult to pass lackluster work through these places Not really that hard. It's difficult to pass outright bad work through a top-tier conference, but above a quality threshold, it's basically a crapshoot; the reviewing process doesn't reliably separate middling from great papers. The very low acceptance rates (~10-15%) and very high inter-reviewer variance [1] combine to make it pretty random whether a given paper, of sufficiently "ok" quality, gets accepted or not. So the main successful strategy to accumulate a lot of papers in top-tier conferences is simply to submit a lot of decent papers to top-tier conferences. How many such papers a given professor can churn out depends largely on how many students, post-docs, and research scientists they can hire, which depends on funding. [1] NIPS, one of the top-tier machine learning conferences, did a very interesting experimental study of this. http://blog.mrtz.org/2014/12/15/the-nips-experiment.html http://blog.mrtz.org/2014/12/15/the-nips-experiment.html

by hayesbh 12y ago

mjn 12y ago

> It's very difficult to pass lackluster work through these places Not really that hard. It's difficult to pass outright bad work through a top-tier conference, but above a quality threshold, it's basically a crapshoot; the reviewing process doesn't reliably separate middling from great papers. The very low acceptance rates (~10-15%) and very high inter-reviewer variance [1] combine to make it pretty random whether a given paper, of sufficiently "ok" quality, gets accepted or not. So the main successful strategy to accumulate a lot of papers in top-tier conferences is simply to submit a lot of decent papers to top-tier conferences. How many such papers a given professor can churn out depends largely on how many students, post-docs, and research scientists they can hire, which depends on funding. [1] NIPS, one of the top-tier machine learning conferences, did a very interesting experimental study of this. http://blog.mrtz.org/2014/12/15/the-nips-experiment.html http://blog.mrtz.org/2014/12/15/the-nips-experiment.html