5 ms·
This is pretty awesome. Clausen seemed a bit snotty about the original theory not being in a peer-reviewed journal. Now the followup is, and the underlying sc
by notacoward 12y ago
This is pretty awesome. Clausen seemed a bit snotty about the original theory not being in a peer-reviewed journal. Now the followup is, and the underlying science also blows a hole in one of Clausen's own papers. He messed with the wrong guy. Citizen science FTW.
- jakevn 12y agoI don't think it's snotty to be skeptical of the results. After all, their original hypothesis turned out to be incorrect. ODAP was not present in the seeds. Because the author took Clausen's consideration seriously, they were able to uncover their mistake and correct it.
- notacoward 12y agoIt's not snotty to be skeptical, but it is snotty to be "skeptical" because the theory comes from an amateur, and to go on about "highly technical and complicated" etc. That's pure appeal to authority, the error of which is amply demonstrated by the fact that one of Clausen's own papers got through peer review with the exact same mistake. Hopefully he has learned to address the science, instead of the source or the publication process, next time.
- jakevn 12y agoIt would, however, be easy for a non-peer reviewed amateur to make a mistake. It's an appeal to authority, but in this case I don't believe it's a fallacy. Furthermore, since peer review is fallible, I don't believe Clausen's identical mistake making it through peer review proves the process any less important. Perhaps Clausen's criticisms were not well intended, but wouldn't any respectable scientist demand rigor before seriously considering such a hypothesis? After all, the burden of proof is not on Clausen.
- notacoward 12y agoThe mirror twins of ad hominem and ad verecundiam (appeal to authority) are always fallacies, regardless of person or circumstances. To say that certain people should get away with them is itself an appeal to authority. In logic, science, or debate, the identity of the person making or disputing a claim is irrelevant. If Clausen had a concern about the data, he should have said so. If he had a concern about the reasoning applied to that data, he should have said so. That would have been "demanding rigor" as you put it, but he did neither. Instead he addressed identity, and that's where he came off as snotty.
- deleted 12y ago[deleted]
- jakevn 12y agoHe did have a concern about the data. His concern that was it was not peer-reviewed. Had the data come from someone with a proven track record in the field, he may have found it worth his time to peer review it himself. Appeal to authority is not always a fallacy. It would have been fallacious if he was outright denying or accepting a hypothesis as being the logical conclusion based on one's standing. All that I read from it, however, was that he had a hard time believing it, and would not spend the time to look into the data personally, but that it should be reviewed. I don't think it's illogical to take more seriously the hypothesis of someone with a proven track record of good research.
- notacoward 12y agoIf he couldn't spend the time to examine it, he should have just said that instead of taking the time to blather about journalistic vs. academic review processes. Clearly he did have the time; he just used it poorly.
- sanderjd 12y agoYour argument is interesting, but it seems to boil down to dismissing the entire system of peer reviewed journals. If they can't be appealed to, and everything needs to be independently researched based on personal expertise, then they have no purpose. Would you agree with that?
- saalweachter 12y agoI'd like to chime in here. The valuable thing isn't individual peer-reviewed papers, it's bodies of peer-reviewed works. "Peer-review" as most people think of it -- the vetting of papers by a few reviewers before publication -- is the start of the process, not the end. Peer-review is what happens after your paper is published and your peers -- the people who read the papers you publish -- read your paper and discuss your paper and try and succeed or fail to replicate your results. A journal isn't "peer-reviewed" because a couple of schmucks read the paper before publication; that's just the scientific equivalent of Fizz-Buzz to avoid wasting everyone's time with obvious crap. It's "peer-reviewed" because it is widely and critically read by a large number of practitioners in a field. You should never regard an individual paper uncritically just because it is published. Later, in light of a larger body of research, you might regard a paper as particularly insightful or seminal.
- lotharbot 12y agoThis is a wonderfully clear explanation. Well said. To generalize a little more: science isn't about single pieces of data or single experiments. Science is about the process. Experiment/data -> idea -> experiment/data -> refined idea -> ... until the idea and the data match to the greatest possible degree (a temporary state of affairs, until science in other areas makes it possible to refine measurements even more tightly.) A single paper, even one with solid methodology, extensive data, and clear patterns, is just one part of science. Replication, refinement, methodological improvements, identification of weaknesses, etc. are what make the whole institution of science so powerful.
- hessenwolf 12y agoI quite enjoyed how the author quoted, “I would be much more convinced if I was reading the report from a credible peer-reviewed professional.”, without criticising the quote themselves. The reader is left to decide on their own that the quote is douchey.
- defen 12y agoThe meat of Clausen's paper isn't really wrong - he just drew the wrong conclusions by making an unwarranted inference. He showed that Hedysarum mackenziei / alpinum do not contain significant levels of alkaloids, and that they do not inhibit bacterial growth. He deduced that this "strongly suggested" they were not toxic. He was wrong because the plants contain a non-alkaloid compound that is toxic to mammals (and presumably not bacteria). So there is a lesson here about what kind of conclusions you can draw from scientific data.
- mcmancini 12y agoI don't think your read on Clausen is accurate. Krakauer's excerpt of Cole's article was poorly done. If you go to the source article, Clausen says, “First, it seems like a rather remarkably lucky shot in the dark for a person to suspect the presence of a specific toxin on so little evidence," he wrote. "Couple this with the observation that ODAP is only reported in certain plants of the genus lathyrus which is not the genus hedysarum and the conclusions are even more remarkable (though not impossible). I would be much more convinced if I was reading the report from a credible peer-reviewed professional journal such as the Journal of Natural Products or equivalent. I have often heard a key indicator of bad science that is highly suspect is having its results published in the popular media." ... "Again, I don't make any claims that the report is wrong since I have no data to analyze, but I am skeptical and will remain so until I see a better forum for the results to be published in." [1] which to me sounds reasonable and, as a professional scientist, sounds just like what I (and I expect, my colleagues) would say. The peer review process isn't perfect, but it is consistently superior to the crap that is scientific journalism appearing outside of peer reviewed journals. Here's why, and why I don't understand why you would hold Krakauer as an example of citizen science. Krakauer made (by his own admission) a "rash intuitive leap" [2] that an unidentified alkaloid contributed to McCandless' death, based on one of McCandless' journal entries and unpublished preliminary research by Clausen. As an aside, if one wants to piss off a scientist, that's a good way to go about it. When Clausen completes his research and finds the mystery alkaloid theory doesn't work, Krakauer intuits that a mystery toxic mold must have been involved, a theory he announces on Oprah and NPR. This second theory doesn't hold up (as was demonstrated by Clausen, again). Despite your conjecture, it sounds like Clausen actually investigated Krakauer's theories, up to a point. Krakauer then latches on to ODAP, which was close but still not there. His fourth theory may be correct, but Krakauer disseminated a lot of incorrect information along the way. I wonder, did he publish retractions as a professional scientist would? I'm not sure why you would hold Krakauer's methodology as "citizen science FTW", but I'm interested to hear. [1]: http://www.adn.com/article/20130917/krakauers-wild-theory-mccandless-gives-short-shrift-science http://www.adn.com/article/20130917/krakauers-wild-theory-mc... [2]: http://www.newyorker.com/books/page-turner/how-chris-mccandless-died http://www.newyorker.com/books/page-turner/how-chris-mccandl...