5 ms·
In small fields, it's sometimes really not very hard to guess who are the authors, just given the topic and what papers they cite. Even with double blind.
by the_svd_doctor 6y ago
In small fields, it's sometimes really not very hard to guess who are the authors, just given the topic and what papers they cite. Even with double blind.
- qznc 6y agoYou can often guess who the reviewers are as well. The most obvious tell is "also cite paper X" which is their own 99% of the time.
- xenonite 6y ago"Please update the references to include recent literature" sometimes makes it clear, too.
- lazyjeff 6y agoI don't think this is true from my experience. I've been on several program committees, and have managed a couple as well. Usually if you're above the first-level reviewers (e.g. senior PC, track chair, PC chair), you can see the actual author names and reviewers. Most of the time when an author later says (formally through the review system, or informally in casual conversation) that "I know it's this reviewer who asked me to cite their own paper" they are wrong. Their main evidence is that the reviewer asked them to cite two papers that include the same author. But of course I can't tell them otherwise because of the confidentiality, so they keep on believing that, perpetuating the myth. Perhaps someone can aggregate some statistics on this, but I genuinely believe that authors suggesting their own papers only about 20% of the time, and when they do it's because their paper was highly relevant.
- ska 6y agoI agree, in my experience the request to add citations is usually because there is fairly obvious context missing, and the reviewer wants to make the connection/comparison is made. Sometimes it's the authors own papers, but that's selection bias at work - if they didn't work in the area, they probably wouldn't be reviewing it.
- zoomablemind 6y agoMakes sense. Also at times the automated anti-plagiarism check systems may list some references by similarity of the topic or the certain degree of match, not necessarily suggesting that it's been copied. So it's helping the reviewer to suggest such references to the relevant extent.
- cycomanic 6y agoI think there are two aspects to this, the first one as you mention is that often reviewers just asked to cite relevant papers (and it's not unusual they come from the same author, because the fields are small). The other aspect is, that there are a some black sheep who will ask you to cite a whole bunch of barely relevant papers (often >5) all from the same group. With those you very clearly see that they are abusing the system. I suspect that these instances cloud people's judgement on the first aspect, so they always interpret the question to cite as fraudulent. As a general advice if you suspect reviewers are try to push their citation counts up, a note to the editor (or PC if a conference) will typically help, because most don't look very favourably at this.
- throwawayiionqz 6y agoIt's even easier to trick the authors into believing you (the reviewer) is someone else. E.g: ask the authors to cite two papers of another researcher (as was mentioned somewhere in the thread as making it easy to guess who the reviewer is), ask to compare to results of that researcher more thoroughly, use some writing style typical of that researcher country of origin, etc. It's especially easy if you are familiar with works from the other researchers for instance because you also reviewed their work.
- deleted 6y ago[deleted]
- m0zg 6y agoNot even in "small" fields. A lot of the time any new work is just a further development of previous work by the same author. So if you're well read in your field, that "double blind" peer review is not blind at all.
- hoistbypetard 6y agoThe cure to that (reviewers who are not well read in their field) seems far worse than the disease.
- m0zg 6y agoI'm not saying we need a cure. I'm just saying we can all stop pretending that those reviews are "double blind" to the extent currently assumed by many.
- sqrt17 6y agothe cure to that is a field where independent people can also contribute to further development of existing work - through making sure that resources used and code are published along with the paper. Of course, this is hard to do with cell cultures and such, and in the case of large databases or compute-intensive tasks it's not quite feasible. But a surprising amount of what goes on in CS and neighbouring disciplines can work that way once you work past the reticence of individual established authors and set it as a goal for your (sub-)field.