5 ms·

How is double blind not the norm? there is just too much room for bias

ghaff 4y ago
Past performance is a reasonable indicator in a lot of situations. I've had this discussion in the context of conference submittals. While there are lots of reasons to be inclusive of new presenters, the reality is that I also want to be aware as a conference committee member of applicants who have been very popular in the past.
- hedora 4y ago
  I've found the opposite. I don't want to hear version 10 of a paper from 2005. Besides, invited speakers are a thing for a reason.
cperciva 4y ago
If you want a review to be truly double blind, you have to start censoring the paper -- things like "in previous work [1-10] we showed..." makes it obvious who the authors are even if you remove the names from the top of the paper.
- trepetti 4y ago
  You do not have to censor: you simply refer to yourselves in the third person to make your shared identity plausibly deniable. A work that is worthy of being of publication should largely be able to stand on its own anyway. This may have the effect of reducing minimum publishable unit (MPU) CV spam papers as a bonus.
  - cperciva 4y ago
    I'm talking about editors censoring papers. Authors could avoid this problem by writing differently, but as a reader I'd prefer they don't; knowing that a body of work is closely related is very useful, especially when it comes to deciding which references to follow up.
    - trepetti 4y ago
      The double blind process only requires that this be done during the review drafts. The final camera-ready version for publication can use the first person without compromising the integrity of the review process. I don't think the editors would remove self-referential text from a final draft after the paper had been accepted.
  - gus_massa 4y ago
    Usually if you count who has more citations you get the author of the paper (or the advisor, or the leader of the team). Sometimes it's a lifehack to get more citations, but most of the times is just natural because someone of the team was working in tool X and other member of the team in tool Y and now you are adjusting all the details to make X and Y work together and get a new result.
- Calavar 4y ago
  There are some computer science conferences that do this. If you cite your prior work, you are supposed to blank out the names in the citation and then attach an anonomyzed version of the prior work so that the reviewers can reference it if necessary. If the paper is accepted, you de-anonomyze the citation in the camera ready version.
  - codethief 4y ago
    This does not necessarily help. In many fields the reviewers will be familiar with the prior work and therefore recognize it, anonymized or not.
- hedora 4y ago
  "This work is based on the ideas of [1-10] which showed..." (Also, it is a major red flag to cite 10 papers from one group and no related work from anyone else. Either the topic is completely irrelevant, or you didn't do a cursory literature search - at least skim the references from papers you cite!)
cozzyd 4y ago
In my field that would be impossible... people work in collaborations and you know pretty much all of the experiments.
- hedora 4y ago
  That's usually a consequence of too much nepotism, or some super expensive piece of equipment. Double blind helps the former, though not the latter. In a part of my field that is capital intensive, some well funded newcomers have recently invested a lot, and "broke in", while some incumbent testbeds went away; the "expensive equipment" problem is usually temporary if the field is expanding.
  - cozzyd 4y ago
    It's just that experiments in astroparticle physics are expensive and have lots of people on them. Like, how is the IceCube Collaboration supposed to write an anonymous paper? Even the most cursory description of the detector would give it away...
    - hedora 4y ago
      Well, maybe there will be many IceCube's later? (Probably not, but I'm routinely shocked when I learn about new giant testbeds in my field...) Alternatively, when IceCube 2 comes out, the old IceCube crowd might be focusing on other stuff, and not paying attention to the IceCube 2 politics. That makes them great peer reviewers (no horse in the race, but knowledgeable).
      - cozzyd 4y ago
        The proposed IceCube Gen2 is mostly a superset and only slightly disjoint with IceCube (for example, I am not an IceCube collaborator but I am on Gen2...). But the point is that for experiments that are larger than a few PI's, anonymity of authors is basically impossible (since all papers have everyone on them).
  - cortesoft 4y ago

Past performance is a reasonable indicator in a lot of situations. I've had this discussion in the context of conference submittals. While there are lots of reasons to be inclusive of new presenters, the reality is that I also want to be aware as a conference committee member of applicants who have been very popular in the past.

hedora 4y ago
I've found the opposite. I don't want to hear version 10 of a paper from 2005. Besides, invited speakers are a thing for a reason.

cperciva 4y ago

If you want a review to be truly double blind, you have to start censoring the paper -- things like "in previous work [1-10] we showed..." makes it obvious who the authors are even if you remove the names from the top of the paper.

trepetti 4y ago
You do not have to censor: you simply refer to yourselves in the third person to make your shared identity plausibly deniable. A work that is worthy of being of publication should largely be able to stand on its own anyway. This may have the effect of reducing minimum publishable unit (MPU) CV spam papers as a bonus.
- cperciva 4y ago
  I'm talking about editors censoring papers. Authors could avoid this problem by writing differently, but as a reader I'd prefer they don't; knowing that a body of work is closely related is very useful, especially when it comes to deciding which references to follow up.
  - trepetti 4y ago
    The double blind process only requires that this be done during the review drafts. The final camera-ready version for publication can use the first person without compromising the integrity of the review process. I don't think the editors would remove self-referential text from a final draft after the paper had been accepted.
- gus_massa 4y ago
  Usually if you count who has more citations you get the author of the paper (or the advisor, or the leader of the team). Sometimes it's a lifehack to get more citations, but most of the times is just natural because someone of the team was working in tool X and other member of the team in tool Y and now you are adjusting all the details to make X and Y work together and get a new result.
Calavar 4y ago
There are some computer science conferences that do this. If you cite your prior work, you are supposed to blank out the names in the citation and then attach an anonomyzed version of the prior work so that the reviewers can reference it if necessary. If the paper is accepted, you de-anonomyze the citation in the camera ready version.
- codethief 4y ago
  This does not necessarily help. In many fields the reviewers will be familiar with the prior work and therefore recognize it, anonymized or not.
hedora 4y ago
"This work is based on the ideas of [1-10] which showed..." (Also, it is a major red flag to cite 10 papers from one group and no related work from anyone else. Either the topic is completely irrelevant, or you didn't do a cursory literature search - at least skim the references from papers you cite!)

cozzyd 4y ago

In my field that would be impossible... people work in collaborations and you know pretty much all of the experiments.

hedora 4y ago
That's usually a consequence of too much nepotism, or some super expensive piece of equipment. Double blind helps the former, though not the latter. In a part of my field that is capital intensive, some well funded newcomers have recently invested a lot, and "broke in", while some incumbent testbeds went away; the "expensive equipment" problem is usually temporary if the field is expanding.
- cozzyd 4y ago
  It's just that experiments in astroparticle physics are expensive and have lots of people on them. Like, how is the IceCube Collaboration supposed to write an anonymous paper? Even the most cursory description of the detector would give it away...
  - hedora 4y ago
    Well, maybe there will be many IceCube's later? (Probably not, but I'm routinely shocked when I learn about new giant testbeds in my field...) Alternatively, when IceCube 2 comes out, the old IceCube crowd might be focusing on other stuff, and not paying attention to the IceCube 2 politics. That makes them great peer reviewers (no horse in the race, but knowledgeable).
    - cozzyd 4y ago
      The proposed IceCube Gen2 is mostly a superset and only slightly disjoint with IceCube (for example, I am not an IceCube collaborator but I am on Gen2...). But the point is that for experiments that are larger than a few PI's, anonymity of authors is basically impossible (since all papers have everyone on them).
- cortesoft 4y ago