3 ms·
To be fair, it's not common to have 9 reviews. OSDI 2012 was particularly thorough (my reviews totaled about ~4000 words) I'm more used to 4-6 reviews, around 3
by arjunnarayan 13y ago
To be fair, it's not common to have 9 reviews. OSDI 2012 was particularly thorough (my reviews totaled about ~4000 words) I'm more used to 4-6 reviews, around 3000 words or so. SOSP/OSDI is probably the most thorough and competently run conference that I know of in all of academia. This makes sense: they only take about 30 papers (it's restricted not just by quality, but the quality bar goes up as more submissions are made, just to keep it exclusive), it's once a year, and systems researchers sync their lives to the OSDI/SOSP submission cycle. It's the "flagship" systems conference, and a lot of advances are first described as OSDI/SOSP papers. For some perspective, the original MapReduce paper was by Jeff Dean and Sanjay Ghemawat, published in OSDI 2004. Last year's best paper was the description of the Spanner Distributed Database by Google.
Incidentally, I have deanonymized several reviews already. When you go to conferences, and some researcher asks the exact same question and has the exact same opinion as reviewer C, chances are... Sometimes the recognition is mutual as they realize what they just revealed (thankfully that has only resulted in humorous chuckles, at least in my life. But I can see how it can go wrong so easily!)
- seanmcdirmid 13y agoYou might only get 9 reviews though if you pass into the second or third rounds. Controversial papers in the PL conferences I haunt will get more reviews out of necessity; better to add more reviewers to help break ties.
- arjunnarayan 13y agoYes. According to the OSDI 2012 proceedings introduction, all papers in the third round had 9 reviews. No paper was accepted or rejected with less than 51% of the PC member's assent.
- seanmcdirmid 13y agoThat is good and bad, it definitely skews toward safe non controversial papers.