9 ms·

Why I retracted my Nature paper: correcting the scientific record

• Gravityloss 13y ago
  While it is dramatic, these kinds of issues are often used by people who want to deny any scientific subject. They can claim that all research is worthless.
  • tokenadult 13y ago
    Deniers like that publish in publications with a lot less integrity and peer review than the journal Nature, so they shouldn't use incidents like this as a basis for their delusional beliefs.
  • glenra 13y ago
    Doesn't that make it even more important that research journals should be transparent and follow their own policies? If top journals were willing to admit mistakes promptly it would be much harder to claim that valid alternative views are being suppressed for political (or other) reasons.
    • Gravityloss 13y ago
      Yes.
  • deleted 13y ago
    [deleted]
  • emerod 13y ago
    I'm not sure I get your point. Are you saying that we shouldn't be discussing problems of scientific methodology, since it strengthens the people who claim there are problems with scientific methodology?
    • Gravityloss 13y ago
      No, I did not mean that. But blowing it all out of proportion is not good either. It was just a reminder that such uses happen. People don't have a balanced exposure to science.
      • emerod 13y ago
        Yes, there is a tendency to believe that the most recent thing published represents "the last word" on the subject. A balanced exposure to science would include an historical view of the contingency of scientific knowledge. But this could only be acquired by reviewing all the published literature.
  • nknighthb 13y ago
    I don't "deny any scientific subject", nor do I think any research is worthless, but over the last few years as these sorts of stories have come to light more, I have become convinced that publication and peer review as currently manifested are worthless.
• deleted 13y ago
  [deleted]
• spartango 13y ago
  Like virtually every other collection of people, the biomedical sciences have their own drama and politics. It is unfortunate that these occasionally impede the progress of science, and downstream, the eventual treatment of patients. Even so, much good work is done and will continue to be done into the future, despite politics.
• lolgay5 13y ago
  Absolutely shameful. Nature editors should step down and issue a public apology immediately.
  • cantankerous 13y ago
    I'd tend to agree if we had some 3rd party verification of the inability to reproduce the results of the initial study. Right now it's somebody vs somebody else.
    • prospero 13y ago
      The article mentions several independent failures to reproduce.
      • dllthomas 13y ago
        Further, "We think it's reproducible" is an absurd defense. If it's reproducible, point to a reproduction. If it hasn't been reproduced, you don't have any clue if it's reproducible.
    • ScottBurson 13y ago
      I agree that some caution seems advisable. What I would really like to see in such cases is not so much other people simply trying to reproduce the results, but an investigation into why one group got different results. Certainly it could be for a scientifically uninteresting reason -- ranging from outright fraud, to sloppy experimental technique causing contamination, to arithmetic errors in analyzing the results -- but it could also be for a scientifically interesting reason: perhaps they used cells from a different supplier that differ in some relevant property that nobody had previously noticed, for example. In short, there might be some interesting discovery to be had, buried underneath the discrepancy. I hate to see us overlook that possibility in our haste to get the "right" answer.
  • duskwuff 13y ago
    Er... it's not like this just happened. The retraction being discussed was in 1998.
• epistasis 13y ago
  There are many well documented cases showing that both Nature and Science are particularly susceptible to these types of shenanigans. I don't know if this is unique to them because of their particular competitive spirit, or if distaste in examination extends to most journals and it's just that Nature and Science see far more scrutiny than other journals. These high profile journals serve the purpose of publicizing discoveries that are thought to have far-reaching interest in science. However, at the same time, they have the strictest limitations on publications, meaning that the methods section, the foundation of science, are rarely described in any detail. So they have some of the worst practices for publication while supposedly chronicling the most important discoveries. So while Nature/Science papers make the news and make people's careers, it's important to downweight the certainty of the finding until there is independent verification.
  • ChuckMcM 13y ago
    This is true. I join AAAS in order to get access to Science and some of the related journals and have become quite disillusioned at their practice of either hiding, or outright omitting, papers or studies that contrast something that appeared 'first' in their pages. I was always taught that a scientist should never fear being wrong, they should only fear being not right. The discourse and disagreements, the questions and reproduction things. make things more real.
  • akiselev 13y ago
    From the horse's mouth: only 6 out of 53 "landmark" oncology studies were significantly reproducible [1]. Bayer did a similar study, found that they couldn't reproduce the results of about 2/3s of the papers they tried to use for drug discovery R&D [2]. This is a near universal consequence of academia's incentives and extends into many of the sciences (although biology is most susceptible due to its complexity). [1] http://www.nature.com/nature/journal/v483/n7391/full/483531a.html http://www.nature.com/nature/journal/v483/n7391/full/483531a... [2] http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.html http://www.nature.com/nrd/journal/v10/n9/full/nrd3439-c1.htm...
    • gizmo686 13y ago
• wallio 13y ago
  Fascinating issue and the rest of the Retraction Watch website is eye opening! I rarely read academic journals anymore but I had always assumed the top journals did a good job of filtering out bad papers and quickly retracting those that were later disproven. Like every human activity it is not so simple. http://retractionwatch.wordpress.com/ http://retractionwatch.wordpress.com/ Edit: Warning this is blog addictive - I just spent a solid hour reading. It's kinda science gossip but still important.
• crocowhile 13y ago
  For those who are not familiar with the topic: retracting a news and views was like giving the journal a (quite deserved) middle finger.
• a3_nm 13y ago
  What I find the most surprising in this story, without having to take sides about the merits of any of the discussed papers, is that the initial refutation of the work of Bellgrau et al. was reviewed by Bellgrau et al. If this fact is true, then there is a clear conflict of interest: Bellgrau et al. should have declared it and declined to review, or, barring this, the editors should have noticed it and ignored the review.
  • ISL 13y ago
    If a paper is being contradicted by another, it's quite common to send a copy for review to the original authors. Who better to look for holes in a new paper than someone who's expert in the field and about to have a big problem on their hands? The editors are the ones who make the go/no-go decision. In my field, I've seen papers that are faulty and have been rejected by experts be accepted by major journals. There's a lot that goes into the go/no-go decision by an editor; I don't envy that job. If the article is correct, then the editors at Nature may have made the wrong decision. Most of us are sufficiently un-expert in the field to make any conclusion.
    • jessriedel 13y ago
      I believe in this case, where two outside reviewers recommended acceptance and the original authors recommended rejection, that the appropriate action for the editors was to get the two outsider reviewer's opinion on the original author's report. Even better, a third outside reviewer should be contacted to assess the original author's report.
      • ISL 13y ago
        I'd even call for greater numbers of referees. There are big practical problems with doing so, but the uncertainty associated with the traditional two or three referees is >50% ( sqrt(3)/3 ). An ensemble of 7-8 referees may produce a more reliable result.
        • jessriedel 13y ago
• rrrrtttt 13y ago
  Science publishing is a classical case of the Market for Lemons [1]. The authors are like used car salesman in that they know so much more about the results in their paper than their readers and referees. Even if just a minority of authors abuses this, the value of publishing a paper plummets and honest authors don't bother any longer with the whole process. The well has been poisoned. [1] http://en.wikipedia.org/wiki/The_Market_for_Lemons http://en.wikipedia.org/wiki/The_Market_for_Lemons
• tokenadult 13y ago
  Now that we have online publishing, and all scientists around the world appear to have email and Web access, we can do much better. Jelte Wicherts is a young professor of psychology who would like to clean up peer review and scientific publication practices in that discipline and other disciplines. In Frontiers of Computational Neuroscience (an open-access journal) he and his co-authors give a set of general suggestions Jelte M. Wicherts, Rogier A. Kievit, Marjan Bakker and Denny Borsboom. Letting the daylight in: reviewing the reviewers and other ways to maximize transparency in science. Front. Comput. Neurosci., 03 April 2012 doi: 10.3389/fncom.2012.00020 http://www.frontiersin.org/Computational_Neuroscience/10.3389/fncom.2012.00020/full http://www.frontiersin.org/Computational_Neuroscience/10.338... on how to make the peer-review process in scientific publishing more reliable. "With the emergence of online publishing, opportunities to maximize transparency of scientific research have grown considerably. However, these possibilities are still only marginally used. We argue for the implementation of (1) peer-reviewed peer review, (2) transparent editorial hierarchies, and (3) online data publication. First, peer-reviewed peer review entails a community-wide review system in which reviews are published online and rated by peers. This ensures accountability of reviewers, thereby increasing academic quality of reviews. Second, reviewers who write many highly regarded reviews may move to higher editorial positions. Third, online publication of data ensures the possibility of independent verification of inferential claims in published papers. This counters statistical errors and overly positive reporting of statistical results. We illustrate the benefits of these strategies by discussing an example in which the classical publication system has gone awry, namely controversial IQ research. We argue that this case would have likely been avoided using more transparent publication practices. We argue that the proposed system leads to better reviews, meritocratic editorial hierarchies, and a higher degree of replicability of statistical analyses." The practices recommended in Wicherts's article would have made an incident like the earlier retraction from Nature reported here much less likely to be necessary in the first place.
• ezy 13y ago
  These journals like to say they need the exclusivity, prestige and money to support their editorial process. Yet, given the exclusivity, money and prestige, they apparently don't do a great job. On the other hand, is there a clear alternative? Is there a way to organize the review process such that the power of the editor is reduced, yet the quality is maintained? I'm not sure. It seems like this would require non-anonymous open review and a some kind of rating/comment system for papers. It's possible that what ends up happening is that it creates more work for the reader in the same way that it takes effort to tease out a products true worth on Amazon. That might not be a bad thing at all.
  • jessriedel 13y ago
    The literature is dauntingly massive, and there is clearly a demand from scientists for an instant method of identifying excellent and important work. Currently, journal name fills this role (however imperfectly). Anonymous review exists for a reason and doesn't seem to have much to do with the first issue you raise: the owners of top journals collect large economic and intellectual rents from their monopoly ownership of the top journal name. I actually don't think the anonymity of the review has much to do with the issue raised by the OP, and I'd suggest putting it aside. So you want some sort of rating system for papers. In the long term, this is usually citation count, but in the short term it probably has to be judged by a trusted human being. Right now, the journal editors fill this role not so much because they are particularly smart, but because their monopoly on the name ensures that all the best work will be sent to them. Note, however, that just asking the smart scientists what they're reading is not enough. Reviewing articles, and coordinating multiple reviews, is hard work. Right now, the editors get compensated in cash (derived mostly on their name monopoly) and the scientists get compensated with the warm fuzzy idea that they helped the community, and maybe with some back scratching on their own paper from the editor in the future. A potential solution needs to figure out which humans are going to be assessing papers for extremely important (which needn't be the same thing as the peer review), how they will be compensated, where the money will come from if they are compensated in cash, how to deal with the perverse effects of cash incentives, and how to prevent any small group of humans from controlling the process. This is not an easy problem.
  • 13y ago
• driverdan 13y ago
  > it showed that expression of CD95L on Sertoli cells in allogeneic mismatched testes tissue transplanted under the kidney capsule was able to induce apoptosis of invading cytotoxic T cells I imagine the way reading this made me feel is similar to how an average person feels reading about complex computer science issues.
  • pjscott 13y ago
    Yes, exactly. It's a pretty straightforward statement -- if you know what the words mean, and why it's significant. If you don't have that background knowledge, it's completely opaque.
  • refurb 13y ago
    Basically it's just saying that if you take a particular genetic sequence and introduce it into foreign cells, you can put them into a body where you'd expect an immune response (t cells), but in fact you actually see those cells die instead (apoptosis). It's too bad it didn't turn out to be true as it would have been an incredible advancement in transplantation.
• danbmil99 13y ago
  Did I read correctly, that the authors of the original paper were allowed to review the rebuttal article, and even worse, their single negative review caused the paper to not be published? Isn't there some concept of recusal, conflict of interest in allowing a researcher whose results are being challenged to review the challenge, and veto its acceptance? That's absurd on its face.