7 ms·

Should scientific papers be anonymous?

70 points by tokenadult 11y ago

peter303 11y ago
Anyone good enough to peer review would probably know who did it. Research areas are often not that large.
- MikeNomad 11y ago
  Yes. When I was doing peer review work in my field (Mass Com), it took about a year to get the landscape sorted. That includes being able to ID people behind submissions, based on writing style. Unfortunately, I saw submissions get down voted not because of the content, but because of some long standing intellectual feud (I found this out after I became a sought-after "tie breaker." The journal editors figured out I was indeed casting votes based solely on the submission in hand).
- cschmidt 11y ago
  Not only that, but you inevitably self-site your earlier work in your later work. In most cases, the most cited author in a paper is the author of the paper itself. So just look at the last page of the paper, and you can probably tell who wrote it.
mynegation 11y ago
Previous discussion on HN: https://news.ycombinator.com/item?id=10762409 https://news.ycombinator.com/item?id=10762409
- tokenadult 11y ago
  That previous discussion is of a different article by a different author, but, yes, this issue deserves a lot of discussion. The article submitted to open this thread is by two authors who follow the research literature on this topic closely and who review much of the prior literature in their commentary article.
  - scott_s 11y ago
    A different article, yes, but I think the currently submitted one is a reaction to the linked one. I was going to link to the previous discussion, too, as I think this discussion and that one are necessarily related. Which makes me think maybe HN should have some mechanism to say "This article is related to this other one", and point to the HN submission for it.
    - tokenadult 11y ago
      Which makes me think maybe HN should have some mechanism to say "This article is related to this other one", and point to the HN submission for it. I guess our user comments here in this subthread are that mechanism. By the way, this whole thread is an implementation of the new HN algorithm that revives threads that are missed on first submission and gain upvotes later. I notice that is implemented by resetting the submission date on the submission. (I was the original submitter, so I remember when I first submitted this thread.)
rubidium 11y ago
At the root of this proposal is the fantasy that scientists would somehow be selfless adherents to the alter of the scientific method. They give up all honor, pride and recognition so that the "scientific method" be protected from the often selfish desires of human beings. In the real world, having names on papers is a really good thing. In real academia, I've heard professors say "well that work was done by so-and-so, and they're bad researchers so I don't trust their results". There's this popular misconception that just because something is published in a peer-reviewed journal that it must be true. Nobody in research actually believes that. All it means is the authors were able to convince the reviewers and the peer reviewers to publish it.
- api 11y ago
  What if papers were cryptographically signed so that they can be claimed later and there were a period of, say, one year of anonymity after publication? This period would allow things like peer review and initial recognition of the paper to occur with less social bias. Just as important would be institutional anonymity. A paper from Nowheresville should be considered equally to a paper from Harvard. Finally I think papers should be required in their headings to disclose sources of funding for the work therein, although this would have to be done later so as not to indirectly compromise the anonymity period.
  - rubidium 11y ago
    What problem are you trying solve? As bad as problems as the academic community has (it has lots!!), names on papers is not one of them. Science is the organic collection of all people and resources and knowledge engaged in figuring out how our world works. Having blind papers would mean I would just have to spend an extra 10 minutes reading each paper to figure out what group wrote it. The fact that Prof. Ivy League gets her results recognized more easily then Prof. Nowhereville is, actually, an ok system. Because Prof. Ivy is more likely to be pursuing interesting problems to the rest of the scientific community, she's more likely to be trusted by other scientists to be a good scientist (i.e. not careless), and the system supports people who have "made it" through the academic gauntlet. If Prof. Nowhereville actually finds something interesting, pursues publication, passes peer review, and it gets published then great. His results are probably worth the attention of the scientific community. But the fact that the barriers are higher is ok, because he hasn't proven himself to be as trustworthy.
cryoshon 11y ago
Wouldn't work. Gotta justify getting your grants by being able to point to previously published work. Plus, everyone can tell who did what-- if not exactly who, then one of a small (5-10) group of people who are likely in the same geographical location.
amelius 11y ago
No, but it would be nice if we had a place where we could discuss scientific papers anonymously. Of course, to keep things concentrated, it would be nice if there was exactly one such place.
- darkmighty 11y ago
  I've always wanted arXiv to fill that role. I understand though that simple anonymous comments don't cut it. It can devolve into youtube comments quickly -- personal attacks to the authors, low effort comments, spamming, "trolling", etc. Those sites need a private reputation system: you can vote comments but the commentator is anonymous; it's hidden reputation should help sorting the comments.
  - fdej 11y ago
    A stackexchange site with an automatically posted "question" for each arxiv submission would be an interesting idea.
- tokenadult 11y ago
  it would be nice if we had a place where we could discuss scientific papers anonymously Have you looked at PubPeer? There have been some very interesting posts there. https://pubpeer.com/ https://pubpeer.com/
rwj 11y ago
While I'm not sure about the complete anonymity suggested by Hanel, I do wish that journals would move to double-blind reviews. It would not be perfect, as often research know enough to guess who the authors might be, but it would still reduce bias.
Fomite 11y ago
One of the journals in my field requires you to strip out all identifiers before submission. This is remarkably difficult. Consider the things that might give you away: - GitHub, FigShare, institutional repositories etc. for code and data - Information on the location of the study - Statistical packages used (hope you didn't author the one you're using) Of course, anonymity will be shattered the moment anyone talks about their work at a conference. Furthermore, several systematic reviews and meta-analysis studies I've worked on benefitted greatly from being able to email the authors directly with questions. I'd also be concerned that people will mentally assign things to a "Big Name" in the field anyway. "Ah, another paper on HAART. Must be out of so-and-so's lab..."
- tibbon 11y ago
  Sure- you can absolutely figure out who wrote the paper given enough work. Yet, I really do like the idea of first the paper being reviewed on it's own merit/ideas/findings, and then potentially people seeing who was involved as a secondary thing at some later point after judgements had already been made.
  - Fomite 11y ago
    I really like double-blind peer review. This was more a note that, when the paper is accepted, those identifying things go back in. The proposal to have papers be anonymous would have to leave them back out, stripping the paper of important context, and making open science a bigger pain than it already is. Which is kinda contrary to the goal.
    - scott_s 11y ago
      I have submitted several papers for double-blind peer review. (I work in the systems side of computer science.) My rule is that I will not weaken a paper through the double-blind process - that is, even if saying this paper builds on prior work may give enough information to out me or my co-authors, I still do it. I still say what system we used, and say it builds on previous work, but I pretend that that work was done by someone else. In practice, someone who really wants to break the double-blind veil probably can. But I don't think that's a problem, as very few people are consciously biased, and will actively seek out such identifying information. By removing names and affiliations, and pretending all of our work was done by someone else, I think that's enough to help people avoid unconscious bias.
smikhanov 11y ago
It sounds like a good idea for the topics that are highly skewed by the political agenda. Like global warming. Imagine a situation when some climatologist makes a discovery that global warming is a fluke. In the current political climate no reputable journal will accept his article on this topic. Moreover, no scientist will even SUBMIT one because of the possible damages to his reputation. On the condition of anonymity, it may stand a chance.
- bitwize 11y ago
  They could do what Satoshi and Bourbaki did: submit pseudonymously.
  - wolfgke 11y ago
    For many (all?) journals, you have sign a consent to publish form. As soon as you sign it, your anonymity is over.
deleted 11y ago
[deleted]
ninguem2 11y ago
How would one decide who gets tenure, promotion and raises if all papers are anonymous? Once universal basic income is implemented, maybe. Meanwhile, no.
adeptus 11y ago
Better idea: Submit it anonymously with your public PKI hash. Then wait X years (5?) and optionally announce yourself by signing the hash to prove you are the author. That way, enough time is given to have the content unbiasedly considered, yet you still get to take credit a bit later.
deleted 11y ago
[deleted]
jpambrun 11y ago
Ironically, none of the authors were selfless enough to submit anonymously.
clarkmoody 11y ago
The landscape of incentives within scientific publishing is complicated. You have to map out what drives each of the parties: journals, authors, reviewers, readers. Journals want high-quality submissions from well-known authors in order to maintain the brand. They also want ongoing subscription fees. Authors want to be published in good journals, but they also want raw publication count to be high. So the primary incentive may not be intellectual honesty. Reviewers often are in competition with the authors of the papers they're reviewing, hence the term peer-review. They have an incentive to prevent publication of competing work. Readers want all knowledge to be free, so they have an incentive to not pay for journals. They also want to know which works are important and which are crock, so name-recognition comes into play. I've probably left something out, but you get the idea. Coming up with new ideas is fun, but until you align the incentives, nothing meaningful will happen. And you've got hundreds of years of inertia and thousands of careers in the balance.
- digbyloftus 11y ago
  "Reviewers often are in competition with the authors of the papers they're reviewing, hence the term peer-review. They have an incentive to prevent publication of competing work." This seems backwards to me. There isn't a fixed amount of interest in a field that is captured by peers. If others in your industry are getting attention then it draws attention to your industry as a whole. I'd think reviewers would be biased towards having competing work published not against.
  - clarkmoody 11y ago
    In some fields, there is extreme pressure to be "first to publish" specific results, and sometimes multiple labs are working on the exact same problem. In this case, if the "peer" is with a competing lab, they are faced with the unethical decision of rejecting the paper in favor of their lab. Or even worse, of stealing the results and trying to publish first.
paulgayham 11y ago
"This paper was published anonymously so it's garbage" and "We'll publish this paper anonymously, clearing our industry of any wrongdoing - and it'll look like good science!"
dekhn 11y ago
The correct term is not anonymized, but de-identified. This wouldn't work in almost any field because the community can already tell with high accuracy who the peer reviewers are, and it's usually easy to tell who wrote a paper by the language style and the conclusions.
platz 11y ago
Should political actions be anonymous? Judicial rulings? Why not anonymize everything we take part in?
ddingus 11y ago
No. Peer review can apply the scientific method to the paper well enough. This may take time. That is just fine. People having their names on papers helps with motivation to publish in most cases, and it's good to understand who contributed what when trying to understand how an idea has formed and who might be with collaborating or talking to. Some topics may be difficult politically. Those are special cases that can and should warrant doing whatever it takes to insure publication and the safety of the scientist. There are also ethics. Anon publishing may well increase questionable research. This should fall into the special case bucket too, as it overlaps with politics.
steinsgate 11y ago
I think the main issue here is the problem of bias in the scientific community. Anonymity is just one proposed solution. There could be others. We should remember that there are three major forms of incentive in the scientific community : 1. Prestige (requires name) 2. Salary and grants (requires credentials, and hence names) 3. The joy of analysis and discovery (doesn't require names) (I might have missed some forms of incentives, but these seem to be the major ones to me.) Anonymity will take the top two incentives away. The second one is a very practical one. There's an existing competition for salary and grants, a pretty brutal one in some areas where money is scarce. We need to solve this problem first before the concept of anonymity can be realistically entertained. I have heard other proposals for reducing bias that seem more tangible. One of them is democratization (partially or fully) of peer review in a centralized publication system (like arxiv).
skynetv2 11y ago
Simple solution is to have blind and double blind reviews. People write papers not only for the greater good but also to improve their careers and profiles. Because thats the only way to get more funding to do more work. By mandating blind reviews, the bias can be removed easily. There are many who give good/bad reviews just by looking at the authors names, not everyone of course. I am not sure why blind reviews are not mandatory by now

70 points by tokenadult 11y ago

peter303 11y ago

Anyone good enough to peer review would probably know who did it. Research areas are often not that large.

MikeNomad 11y ago
Yes. When I was doing peer review work in my field (Mass Com), it took about a year to get the landscape sorted. That includes being able to ID people behind submissions, based on writing style. Unfortunately, I saw submissions get down voted not because of the content, but because of some long standing intellectual feud (I found this out after I became a sought-after "tie breaker." The journal editors figured out I was indeed casting votes based solely on the submission in hand).
cschmidt 11y ago
Not only that, but you inevitably self-site your earlier work in your later work. In most cases, the most cited author in a paper is the author of the paper itself. So just look at the last page of the paper, and you can probably tell who wrote it.

mynegation 11y ago

Previous discussion on HN: https://news.ycombinator.com/item?id=10762409 https://news.ycombinator.com/item?id=10762409

tokenadult 11y ago
That previous discussion is of a different article by a different author, but, yes, this issue deserves a lot of discussion. The article submitted to open this thread is by two authors who follow the research literature on this topic closely and who review much of the prior literature in their commentary article.
- scott_s 11y ago
  A different article, yes, but I think the currently submitted one is a reaction to the linked one. I was going to link to the previous discussion, too, as I think this discussion and that one are necessarily related. Which makes me think maybe HN should have some mechanism to say "This article is related to this other one", and point to the HN submission for it.
  - tokenadult 11y ago
    Which makes me think maybe HN should have some mechanism to say "This article is related to this other one", and point to the HN submission for it. I guess our user comments here in this subthread are that mechanism. By the way, this whole thread is an implementation of the new HN algorithm that revives threads that are missed on first submission and gain upvotes later. I notice that is implemented by resetting the submission date on the submission. (I was the original submitter, so I remember when I first submitted this thread.)

rubidium 11y ago

At the root of this proposal is the fantasy that scientists would somehow be selfless adherents to the alter of the scientific method. They give up all honor, pride and recognition so that the "scientific method" be protected from the often selfish desires of human beings. In the real world, having names on papers is a really good thing. In real academia, I've heard professors say "well that work was done by so-and-so, and they're bad researchers so I don't trust their results". There's this popular misconception that just because something is published in a peer-reviewed journal that it must be true. Nobody in research actually believes that. All it means is the authors were able to convince the reviewers and the peer reviewers to publish it.

api 11y ago
What if papers were cryptographically signed so that they can be claimed later and there were a period of, say, one year of anonymity after publication? This period would allow things like peer review and initial recognition of the paper to occur with less social bias. Just as important would be institutional anonymity. A paper from Nowheresville should be considered equally to a paper from Harvard. Finally I think papers should be required in their headings to disclose sources of funding for the work therein, although this would have to be done later so as not to indirectly compromise the anonymity period.
- rubidium 11y ago
  What problem are you trying solve? As bad as problems as the academic community has (it has lots!!), names on papers is not one of them. Science is the organic collection of all people and resources and knowledge engaged in figuring out how our world works. Having blind papers would mean I would just have to spend an extra 10 minutes reading each paper to figure out what group wrote it. The fact that Prof. Ivy League gets her results recognized more easily then Prof. Nowhereville is, actually, an ok system. Because Prof. Ivy is more likely to be pursuing interesting problems to the rest of the scientific community, she's more likely to be trusted by other scientists to be a good scientist (i.e. not careless), and the system supports people who have "made it" through the academic gauntlet. If Prof. Nowhereville actually finds something interesting, pursues publication, passes peer review, and it gets published then great. His results are probably worth the attention of the scientific community. But the fact that the barriers are higher is ok, because he hasn't proven himself to be as trustworthy.

cryoshon 11y ago

Wouldn't work. Gotta justify getting your grants by being able to point to previously published work. Plus, everyone can tell who did what-- if not exactly who, then one of a small (5-10) group of people who are likely in the same geographical location.

amelius 11y ago

No, but it would be nice if we had a place where we could discuss scientific papers anonymously. Of course, to keep things concentrated, it would be nice if there was exactly one such place.

darkmighty 11y ago
I've always wanted arXiv to fill that role. I understand though that simple anonymous comments don't cut it. It can devolve into youtube comments quickly -- personal attacks to the authors, low effort comments, spamming, "trolling", etc. Those sites need a private reputation system: you can vote comments but the commentator is anonymous; it's hidden reputation should help sorting the comments.
- fdej 11y ago
  A stackexchange site with an automatically posted "question" for each arxiv submission would be an interesting idea.
tokenadult 11y ago
it would be nice if we had a place where we could discuss scientific papers anonymously Have you looked at PubPeer? There have been some very interesting posts there. https://pubpeer.com/ https://pubpeer.com/

rwj 11y ago

While I'm not sure about the complete anonymity suggested by Hanel, I do wish that journals would move to double-blind reviews. It would not be perfect, as often research know enough to guess who the authors might be, but it would still reduce bias.

Fomite 11y ago

One of the journals in my field requires you to strip out all identifiers before submission. This is remarkably difficult. Consider the things that might give you away: - GitHub, FigShare, institutional repositories etc. for code and data - Information on the location of the study - Statistical packages used (hope you didn't author the one you're using) Of course, anonymity will be shattered the moment anyone talks about their work at a conference. Furthermore, several systematic reviews and meta-analysis studies I've worked on benefitted greatly from being able to email the authors directly with questions. I'd also be concerned that people will mentally assign things to a "Big Name" in the field anyway. "Ah, another paper on HAART. Must be out of so-and-so's lab..."

tibbon 11y ago
Sure- you can absolutely figure out who wrote the paper given enough work. Yet, I really do like the idea of first the paper being reviewed on it's own merit/ideas/findings, and then potentially people seeing who was involved as a secondary thing at some later point after judgements had already been made.
- Fomite 11y ago
  I really like double-blind peer review. This was more a note that, when the paper is accepted, those identifying things go back in. The proposal to have papers be anonymous would have to leave them back out, stripping the paper of important context, and making open science a bigger pain than it already is. Which is kinda contrary to the goal.
  - scott_s 11y ago
    I have submitted several papers for double-blind peer review. (I work in the systems side of computer science.) My rule is that I will not weaken a paper through the double-blind process - that is, even if saying this paper builds on prior work may give enough information to out me or my co-authors, I still do it. I still say what system we used, and say it builds on previous work, but I pretend that that work was done by someone else. In practice, someone who really wants to break the double-blind veil probably can. But I don't think that's a problem, as very few people are consciously biased, and will actively seek out such identifying information. By removing names and affiliations, and pretending all of our work was done by someone else, I think that's enough to help people avoid unconscious bias.

smikhanov 11y ago

It sounds like a good idea for the topics that are highly skewed by the political agenda. Like global warming. Imagine a situation when some climatologist makes a discovery that global warming is a fluke. In the current political climate no reputable journal will accept his article on this topic. Moreover, no scientist will even SUBMIT one because of the possible damages to his reputation. On the condition of anonymity, it may stand a chance.

bitwize 11y ago
They could do what Satoshi and Bourbaki did: submit pseudonymously.
- wolfgke 11y ago
  For many (all?) journals, you have sign a consent to publish form. As soon as you sign it, your anonymity is over.

deleted 11y ago

[deleted]

ninguem2 11y ago

How would one decide who gets tenure, promotion and raises if all papers are anonymous? Once universal basic income is implemented, maybe. Meanwhile, no.

adeptus 11y ago

Better idea: Submit it anonymously with your public PKI hash. Then wait X years (5?) and optionally announce yourself by signing the hash to prove you are the author. That way, enough time is given to have the content unbiasedly considered, yet you still get to take credit a bit later.

deleted 11y ago

[deleted]

jpambrun 11y ago

Ironically, none of the authors were selfless enough to submit anonymously.

clarkmoody 11y ago

The landscape of incentives within scientific publishing is complicated. You have to map out what drives each of the parties: journals, authors, reviewers, readers. Journals want high-quality submissions from well-known authors in order to maintain the brand. They also want ongoing subscription fees. Authors want to be published in good journals, but they also want raw publication count to be high. So the primary incentive may not be intellectual honesty. Reviewers often are in competition with the authors of the papers they're reviewing, hence the term peer-review. They have an incentive to prevent publication of competing work. Readers want all knowledge to be free, so they have an incentive to not pay for journals. They also want to know which works are important and which are crock, so name-recognition comes into play. I've probably left something out, but you get the idea. Coming up with new ideas is fun, but until you align the incentives, nothing meaningful will happen. And you've got hundreds of years of inertia and thousands of careers in the balance.

digbyloftus 11y ago
"Reviewers often are in competition with the authors of the papers they're reviewing, hence the term peer-review. They have an incentive to prevent publication of competing work." This seems backwards to me. There isn't a fixed amount of interest in a field that is captured by peers. If others in your industry are getting attention then it draws attention to your industry as a whole. I'd think reviewers would be biased towards having competing work published not against.
- clarkmoody 11y ago
  In some fields, there is extreme pressure to be "first to publish" specific results, and sometimes multiple labs are working on the exact same problem. In this case, if the "peer" is with a competing lab, they are faced with the unethical decision of rejecting the paper in favor of their lab. Or even worse, of stealing the results and trying to publish first.

paulgayham 11y ago

"This paper was published anonymously so it's garbage" and "We'll publish this paper anonymously, clearing our industry of any wrongdoing - and it'll look like good science!"

dekhn 11y ago

The correct term is not anonymized, but de-identified. This wouldn't work in almost any field because the community can already tell with high accuracy who the peer reviewers are, and it's usually easy to tell who wrote a paper by the language style and the conclusions.

platz 11y ago

Should political actions be anonymous? Judicial rulings? Why not anonymize everything we take part in?

ddingus 11y ago

No. Peer review can apply the scientific method to the paper well enough. This may take time. That is just fine. People having their names on papers helps with motivation to publish in most cases, and it's good to understand who contributed what when trying to understand how an idea has formed and who might be with collaborating or talking to. Some topics may be difficult politically. Those are special cases that can and should warrant doing whatever it takes to insure publication and the safety of the scientist. There are also ethics. Anon publishing may well increase questionable research. This should fall into the special case bucket too, as it overlaps with politics.

steinsgate 11y ago

I think the main issue here is the problem of bias in the scientific community. Anonymity is just one proposed solution. There could be others. We should remember that there are three major forms of incentive in the scientific community : 1. Prestige (requires name) 2. Salary and grants (requires credentials, and hence names) 3. The joy of analysis and discovery (doesn't require names) (I might have missed some forms of incentives, but these seem to be the major ones to me.) Anonymity will take the top two incentives away. The second one is a very practical one. There's an existing competition for salary and grants, a pretty brutal one in some areas where money is scarce. We need to solve this problem first before the concept of anonymity can be realistically entertained. I have heard other proposals for reducing bias that seem more tangible. One of them is democratization (partially or fully) of peer review in a centralized publication system (like arxiv).

skynetv2 11y ago

Simple solution is to have blind and double blind reviews. People write papers not only for the greater good but also to improve their careers and profiles. Because thats the only way to get more funding to do more work. By mandating blind reviews, the bias can be removed easily. There are many who give good/bad reviews just by looking at the authors names, not everyone of course. I am not sure why blind reviews are not mandatory by now