5 ms·
In many such events all participants are required to be part of the peer review pool. However, they review a limited amount of papers (e.g. 3) - "everybody vot
by PeterisP 5y ago
In many such events all participants are required to be part of the peer review pool.
However, they review a limited amount of papers (e.g. 3) - "everybody votes" presumes that everybody has an opinion on the rating of every paper. That does not scale - getting a reasonable opinion about a random paper, i.e. reviewing it, takes significant effort; an event may have 1000 or 10000 papers, having every participant review 3 papers is already a significant amount of work, and getting much more "votes" than that for every paper is impractical.
It's unfeasable and even undesirable for everyone to even skim all the submitted papers in their subfield - one big purpose of peer review is to filter out papers so that everyone else can focus on reading only a smaller selection of best papers instead of sifting through everything submitted. The deluge of papers (even "diarrhea of papers" as called in a lecture linked in another comment) is a real problem, I'm a full-time researcher and I still barely have time to read only a fraction of what's getting written.
- foota 5y agoIn theory you could probably do something like have three runoff rounds, such that low-scoring papers are eliminated before people do their second review.
- sjg007 5y agoI disagree plus meta-moderation might help. Then again we see voting rings in HN... but a conference has an entrance fee so maybe that would limit it.