4 ms·

Examine the facts: - Many research fields have anywhere between 3 and 10 groups working in them. - A review always transpires the background of the reviewer.

Examine the facts: - Many research fields have anywhere between 3 and 10 groups working in them. - A review always transpires the background of the reviewer. You just cannot "mask" the shape of your knowledge around a highly-specialized subject. This includes your approach to the problem, the issues you are most interested in (and hence know more about), the references you give, etc. With a closed system, you only get to see: - Reviews of your own paper, without knowing who wrote them. - Reviews of the papers you review (there are usually 3-4 reviewers per paper, and you get to see the other reviewer's reviews and who they are). With these pieces of information, academia is already full of grudges and strong-arming around. Here's an anecdote: I was once in a conference, and met a colleague that made the effort to approach me and comment that he was great friends with my advisor (who didn't attend), and just wouldn't stop praising him. After a while, he switched to ranting about the review process, and how a specific reviewer was a moron that wouldn't understand anything and so on. Of course, that reviewer was my advisor and I did know but he obviously didn't. Given the high egos involved in academia, I am pretty sure that if there was a track record of all reviews, researchers would figure out who reviewed their papers. The backlash would then be ugly, and the entire ecosystem would end up more corrupt that it already is (imho). I've been on all sides of the fence, and I just don't see a better solution than just having secret reviews. Nice words don't matter (much, to most people) when the e-mail starts reading "We are sorry to inform you...".

guscost 8y ago
In the current system, when you get to see the reviews of your paper, there are still the same challenges for anonymity and ugly consequences that you’re pointing out. Perhaps even to a greater degree, because the information is “leaked” through gossip rather than in a formal process. If all reviewer comments were collected and then published at the same time to prevent them influencing each other, how could that make the situation with respect to “frank and honest” reviews any worse than it is already? Are there entirely secret journals where nothing is accessible to non-contributors, for the same reasons? I suspect the answer is no, which is why these feuds often play out in attack-counterattack sequences of published papers.
- kilburn 8y ago
  > So when you see the reviews of your own paper, there are the exact same challenges for anonymity and ugly consequences as you’re pointing out. Yes, and this already happens (hence the grudges I was commenting about). > How does maintaining secrecy for the per-paper improve the situation with respect to “frank and honest” reviews? By increasing the uncertainty. If I am 95% confident it is you who screwed me last year and 50% certain this paper I got now is yours, I will look at it with a harsher attitude than if I'm only 20% and 20% certain. If you were to publish all reviews then researchers would have much more information to convince themselves that it indeed is "that guy". It is sad, and I personally despise it, but that's what it is from my personal experience. PS: I quit academia after getting the PhD, and this was among the reasons for me to quit. The other major reasons being that I don't want to keep relocating somewhere else in the world every 2 years until I'm in my 40s-50s, I don't like the overselling and result dishonesty in general, and I actually enjoy working in industry too).
  - guscost 8y ago
    > By increasing the uncertainty. If I am 95% confident it is you who screwed me last year and 50% certain this paper I got now is yours, I will look at it with a harsher attitude than if I'm only 20% and 20% certain. Ouch, that’s a good (and depressing) point.

guscost 8y ago

In the current system, when you get to see the reviews of your paper, there are still the same challenges for anonymity and ugly consequences that you’re pointing out. Perhaps even to a greater degree, because the information is “leaked” through gossip rather than in a formal process. If all reviewer comments were collected and then published at the same time to prevent them influencing each other, how could that make the situation with respect to “frank and honest” reviews any worse than it is already? Are there entirely secret journals where nothing is accessible to non-contributors, for the same reasons? I suspect the answer is no, which is why these feuds often play out in attack-counterattack sequences of published papers.

kilburn 8y ago
> So when you see the reviews of your own paper, there are the exact same challenges for anonymity and ugly consequences as you’re pointing out. Yes, and this already happens (hence the grudges I was commenting about). > How does maintaining secrecy for the per-paper improve the situation with respect to “frank and honest” reviews? By increasing the uncertainty. If I am 95% confident it is you who screwed me last year and 50% certain this paper I got now is yours, I will look at it with a harsher attitude than if I'm only 20% and 20% certain. If you were to publish all reviews then researchers would have much more information to convince themselves that it indeed is "that guy". It is sad, and I personally despise it, but that's what it is from my personal experience. PS: I quit academia after getting the PhD, and this was among the reasons for me to quit. The other major reasons being that I don't want to keep relocating somewhere else in the world every 2 years until I'm in my 40s-50s, I don't like the overselling and result dishonesty in general, and I actually enjoy working in industry too).
- guscost 8y ago
  > By increasing the uncertainty. If I am 95% confident it is you who screwed me last year and 50% certain this paper I got now is yours, I will look at it with a harsher attitude than if I'm only 20% and 20% certain. Ouch, that’s a good (and depressing) point.