4 ms·
>The result was that almost 60% of papers accepted by one program committee were rejected by the other, suggesting that the fate of many papers is determined by
by sjg007 5y ago
>The result was that almost 60% of papers accepted by one program committee were rejected by the other, suggesting that the fate of many papers is determined by the specifics of the reviewers selected and not just the inherent value of the work itself.
I referenced Kahneham's latest book, Noise, above but this is exactly the problem he focuses on. There are solutions.
- cortesoft 5y agoWouldn’t a ‘solution’ only work if there actually is some underlying objective quality to a paper? It might be the case that different reviewers disagree because there is no right answer, because there is no such thing as objective quality.
- sjg007 5y agoWell wisdom of the crowds suggest that there is a truth... and a conference is charged with representing the best so I think that randomization and meta moderation + other strategies are better in the long run.