3 ms·

> It blows my mind that peer reviews aren't done blind. I've co-authored a few papers and blind reviews have some surprising consequences, like discussions alo

by hannob 4y ago

> It blows my mind that peer reviews aren't done blind. I've co-authored a few papers and blind reviews have some surprising consequences, like discussions along the lines of "you can't say that you already wrote about this issue elsewhere and put a link in the paper, because that would unblind it". I was a bit uncomfortable with that, because I like to "cite my sources" (even if the source was myself in this case). This also points to another issue: The more specialized the paper is the less likely the blinding will work. If you know a field by heart then you know who works on what, and probably can guess most authors based on that. I'm not saying I'm against blind review, but while it sounds obvious, it has some issues in practice.

jedberg 4y ago
Whoever told you that about your cites is probably right -- you shouldn't cite yourself while calling out that you are citing yourself. You should cite yourself the same you that site any other work. The second part is probably unavoidable. If you're working on something super specialized when all your reviewers are the six other people who work on it then sure, it'll be unblinded. Not much we can do there sadly.
- cycomanic 4y ago
  I disagree, the style to write as a "neutral" observer (often writing in passive voice) is frowned upon nowadays for good reason. Part of that is also that the information if the authors wrote a citation or not can be important to a reviewer. For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications. There can be a significant difference in impact between a citation saying "this is important to work on" which is written by the authors and one which is written by someone else. Generally we should not write to hide information, and that would include if the authors wrote other work that relates to the work being reviewed. We should not adjust our writing to doubleblind review (and I would argue the advice the author was given is wrong). Doubleblind review is imperfect anyway, I can often tell who the authors are just by topic and e.g. writing and figure styles, so if a reviewer really wants to know the authors they can. We should still do double blind though.
  - probably_wrong 4y ago
    > the style to write as a "neutral" observer (often writing in passive voice) is frowned upon nowadays for good reason. Could you be more specific about who's frowning upon it? Because I've never heard this before in my field (Comp. Ling., where double blind is the rule) and would like to look more into it.
    - cycomanic 4y ago
      Many style guides now say to write in active voice (Nature is one of them, but many others as well). I don't have the books in front of me, so can't find the citation, but many publications on scientific writing essentially recommend direct language. The reasoning is that the work was "subjective", i.e. carried out by you. By using "detached third person language" you are trying to give a false impression of objectivity. This is similar to management/PR double speak like "we are forced to raise our prices", "we are unable to compensate you"... (I don't assign malice in the case of scientists though).
  - mkmk3 4y ago
    Like sibling I'd also like more information on this and can't find anything immediately compelling with a light search. Like this post suggests, obviously there's a use to hiding information, sometimes a bad reputation is undeserved or irrelevant to the work, sometimes there are subtle biases in play. The reputation of the author and their behaviour wrt the citations used should be considered, I agree with you that it's important information. Maybe the reviewer of an individual paper shouldn't be considering that though, maybe that should primarily be considered in a second review stage or in the context of meta-reviews? Idk, but the spirit of using passive voice in the context of research makes more sense to me.
  - lou1306 4y ago
    > For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications Why is this unfortunate? I'd argue that splitting results in multiple publications is a) Riskier for authors (higher chance of rejection) and b) More convenient for readers (each paper requires less mental load, being focused on a single aspect). So, even if there's a payoff for authors, it doesn't come for free.
- PeterisP 4y ago
  In some cases the issue that you can't really "blind" it because it is a physical continuation of the earlier work - not just building upon the idea, which anyone could do, but using the exact same experimental device (a unique one, like the hadron collider) or continued analysis of the exact same set of patients or animals, or improving the previously made software tool.
- joxel 4y ago
  This happens in larger fields too. For example in AI, which is a large research field, you can be blind-reviewing a paper, but it uses a proprietary google dataset, so you know where it comes from. Blind review is not the answer, I fear.
  - Al-Khwarizmi 4y ago
    For me the most important advantage of blind review is not that 100% of papers are effectively blind, but that if you're a noname author you have the right to be blind and not having your paper looked down on just for that reason. That alone justifies double-blind review. Regarding papers that cite proprietary datasets that no one can access, in fields like AI where it is perfectly possible to release datasets (if there's a specific reason it's a different issue), as far as I'm concerned they should be outright rejected due to lack of reproducibility and inability of the reviewers to check the correctness of the claims. Although I know this is a minority viewpoint and it won't happen.
gms7777 4y ago
Another factor in my field as well is linking to software -- papers that introduce algorithms or use computational analyses are rightfully expected to include the code (usually github) and reviewers are expected to at least check that the code exists and is reasonably documented. I can remove my name from the paper, but the github page will say which lab it's coming from.

by hannob 4y ago

jedberg 4y ago

Whoever told you that about your cites is probably right -- you shouldn't cite yourself while calling out that you are citing yourself. You should cite yourself the same you that site any other work. The second part is probably unavoidable. If you're working on something super specialized when all your reviewers are the six other people who work on it then sure, it'll be unblinded. Not much we can do there sadly.

cycomanic 4y ago
I disagree, the style to write as a "neutral" observer (often writing in passive voice) is frowned upon nowadays for good reason. Part of that is also that the information if the authors wrote a citation or not can be important to a reviewer. For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications. There can be a significant difference in impact between a citation saying "this is important to work on" which is written by the authors and one which is written by someone else. Generally we should not write to hide information, and that would include if the authors wrote other work that relates to the work being reviewed. We should not adjust our writing to doubleblind review (and I would argue the advice the author was given is wrong). Doubleblind review is imperfect anyway, I can often tell who the authors are just by topic and e.g. writing and figure styles, so if a reviewer really wants to know the authors they can. We should still do double blind though.
- probably_wrong 4y ago
  > the style to write as a "neutral" observer (often writing in passive voice) is frowned upon nowadays for good reason. Could you be more specific about who's frowning upon it? Because I've never heard this before in my field (Comp. Ling., where double blind is the rule) and would like to look more into it.
  - cycomanic 4y ago
    Many style guides now say to write in active voice (Nature is one of them, but many others as well). I don't have the books in front of me, so can't find the citation, but many publications on scientific writing essentially recommend direct language. The reasoning is that the work was "subjective", i.e. carried out by you. By using "detached third person language" you are trying to give a false impression of objectivity. This is similar to management/PR double speak like "we are forced to raise our prices", "we are unable to compensate you"... (I don't assign malice in the case of scientists though).
- mkmk3 4y ago
  Like sibling I'd also like more information on this and can't find anything immediately compelling with a light search. Like this post suggests, obviously there's a use to hiding information, sometimes a bad reputation is undeserved or irrelevant to the work, sometimes there are subtle biases in play. The reputation of the author and their behaviour wrt the citations used should be considered, I agree with you that it's important information. Maybe the reviewer of an individual paper shouldn't be considering that though, maybe that should primarily be considered in a second review stage or in the context of meta-reviews? Idk, but the spirit of using passive voice in the context of research makes more sense to me.
- lou1306 4y ago
  > For example it is unfortunately quite common that authors publish results in a salami tactic to maximize the number of publications Why is this unfortunate? I'd argue that splitting results in multiple publications is a) Riskier for authors (higher chance of rejection) and b) More convenient for readers (each paper requires less mental load, being focused on a single aspect). So, even if there's a payoff for authors, it doesn't come for free.
PeterisP 4y ago
In some cases the issue that you can't really "blind" it because it is a physical continuation of the earlier work - not just building upon the idea, which anyone could do, but using the exact same experimental device (a unique one, like the hadron collider) or continued analysis of the exact same set of patients or animals, or improving the previously made software tool.
joxel 4y ago
This happens in larger fields too. For example in AI, which is a large research field, you can be blind-reviewing a paper, but it uses a proprietary google dataset, so you know where it comes from. Blind review is not the answer, I fear.
- Al-Khwarizmi 4y ago
  For me the most important advantage of blind review is not that 100% of papers are effectively blind, but that if you're a noname author you have the right to be blind and not having your paper looked down on just for that reason. That alone justifies double-blind review. Regarding papers that cite proprietary datasets that no one can access, in fields like AI where it is perfectly possible to release datasets (if there's a specific reason it's a different issue), as far as I'm concerned they should be outright rejected due to lack of reproducibility and inability of the reviewers to check the correctness of the claims. Although I know this is a minority viewpoint and it won't happen.

gms7777 4y ago

Another factor in my field as well is linking to software -- papers that introduce algorithms or use computational analyses are rightfully expected to include the code (usually github) and reviewers are expected to at least check that the code exists and is reasonably documented. I can remove my name from the paper, but the github page will say which lab it's coming from.