3 ms·

Yeah, I agree there's a big difference. What reminded me of it is that making finer distinctions get more error-prone as you try to take "the best of the best o

by mjn 14y ago

Yeah, I agree there's a big difference. What reminded me of it is that making finer distinctions get more error-prone as you try to take "the best of the best of the best" and add on more "of the bests". CHI is one of the conferences where I think the low acceptance rate tends towards: 1) a lot of randomness in the reviews; and 2) a tendency towards safe "I did a user study and here are p-values" type of work. I agree there are some times when it's easy to accept only 10% or so: when there is a lot of obvious crap submitted. I'm currently organizing a conference with about a 30% acceptance rate, and it's somewhat painful, though. It's not one of the main catch-all conferences like OSDI, CHI, AAAI, SIGGRAPH, etc. which everyone submits to. Instead mostly people in the area submit to it, and a large proportion of the submissions are legitimately good. I would say 50% solid submissions from regular members of the community, 25% not-great submissions from people legitimately in the area, and 25% either crap or clearly off-topic. So it'd be easy to choose which 50% to accept, but when you're accepting 30%, now some good work is being rejected, just because you have to hit the arbitrary acceptance-rate target to count as "selective" in certain U.S. universities' bibliometric systems. If we went down to a 20% acceptance rate (as some people are pushing, not b/c of quality but solely b/c it makes the conference look better bibliometrically), we'd have to fairly arbitrarily reject some more good papers. We half-joke that the way to fix it next year is to spam the CfP to clearly unsuitable places, in order to elicit a flood of crap submissions that we can then easily reject, allowing us to hit a 30% (or 20%, or lower) acceptance rate without having to reject any good papers.

seanmcdirmid 14y ago
My point with CHI is that they probably admit too many papers, or at least are very unfocused. OSDI and SIGGRAPH have high bars (not sure about AAAI), they don't so much as have a paper limit but just very high standards. > Instead mostly people in the area submit to it, and a large proportion of the submissions are legitimately good. Working groups or invite only workshops might work better in this regard. It depends on your area though. > So it'd be easy to choose which 50% to accept, but when you're accepting 30%, now some good work is being rejected, just because you have to hit the arbitrary acceptance-rate target to count as "selective" in certain U.S. universities' bibliometric systems. Its more about how tenure cases work in universities. Your judges don't read your papers, they just look at your bib. > If we went down to a 20% acceptance rate, we'd have to fairly arbitrarily reject some more good papers. In my reviewing experience, the PC never had to reject papers based on too many good papers; if anything we had to accept papers of questionable quality to fill the conference up. The rejection notices that PC chairs send out about there being too many good papers are often lies. But maybe our fields have different problems :) I'm a firm believer in accepting all the good papers and rejecting all the bad ones. > We half-joke that the way to fix it next year is to spam the CfP to clearly unsuitable places, in order to elicit a flood of crap submissions that we can then easily reject, allowing us to hit a 30% (or 20%, or lower) acceptance rate without having to reject any good papers. Just use Weibo to advertise your conference. There is a lot of talent in China and India who want to make it big, not all of it is great yet, but it will definitely push down your accept rates.

by mjn 14y ago

seanmcdirmid 14y ago

My point with CHI is that they probably admit too many papers, or at least are very unfocused. OSDI and SIGGRAPH have high bars (not sure about AAAI), they don't so much as have a paper limit but just very high standards. > Instead mostly people in the area submit to it, and a large proportion of the submissions are legitimately good. Working groups or invite only workshops might work better in this regard. It depends on your area though. > So it'd be easy to choose which 50% to accept, but when you're accepting 30%, now some good work is being rejected, just because you have to hit the arbitrary acceptance-rate target to count as "selective" in certain U.S. universities' bibliometric systems. Its more about how tenure cases work in universities. Your judges don't read your papers, they just look at your bib. > If we went down to a 20% acceptance rate, we'd have to fairly arbitrarily reject some more good papers. In my reviewing experience, the PC never had to reject papers based on too many good papers; if anything we had to accept papers of questionable quality to fill the conference up. The rejection notices that PC chairs send out about there being too many good papers are often lies. But maybe our fields have different problems :) I'm a firm believer in accepting all the good papers and rejecting all the bad ones. > We half-joke that the way to fix it next year is to spam the CfP to clearly unsuitable places, in order to elicit a flood of crap submissions that we can then easily reject, allowing us to hit a 30% (or 20%, or lower) acceptance rate without having to reject any good papers. Just use Weibo to advertise your conference. There is a lot of talent in China and India who want to make it big, not all of it is great yet, but it will definitely push down your accept rates.