2 ms·
I'm a toss up between blind reviews. I like them in theory but I'm not convinced they work well in practice and at scale I think weird things happen. Review wo
by godelski 2y ago
I'm a toss up between blind reviews. I like them in theory but I'm not convinced they work well in practice and at scale I think weird things happen.
Review works well at small scales because people are more passionate about the work than the money or career, though those matter. There's better accountability in smaller communities because they're more reliant. But blind is often a farce there since you recognize one anothers writings, subtopics, niche, whatever.
At scale review sucks because there is no accountability. Venues need to reject on percentage of submissions, not on the quality of work that year. It's insane it's a zero sum game. And at scale everyone is over burdened and quality control goes down, so malicious/lazy reviewers are even incentivized since they gain time for being lazy, they have less scrutiny when rejecting, and they increase the likelihood of their work being admitted when doing do (I mean every work can be legitimately criticized, so a reject is ALWAYS defendable, even if seen as harsh. This isn't true for accepts). But also at scale, blind is weird. It exists for small labs but not for big ones.
So I don't know. I think many forgot the point of review. That we should all be on the same team (team science), and competition should be friendly. That reviews should leave the authors feeling like they know what they can improve on not like a review read a different paper or just read one figure (and wrong), and just dismissed (that's not a review). And that wastes lots of time and money as well as increases the problem as papers get recycled through the slot machine.
So I don't know. But I do know the point of publishing is to communicate with peers. And this can be done by open punishing. Sure, this might make the signal of citations and number of papers noisier, but I think it's already pretty noisy even if we don't want to admit it. And maybe it's not a bad thing for those to be noisy. I think we rely too heavily on them currently. Maybe if it was clearer that noise existed we could be more nuanced in our evaluations, which is what's frequently lacking these days. Who knows. But I do know the system today is not one I want to continue being in and this sentiment is far from uncommon. So something needs to happen.