4 ms·

"Did you read the original article or just the news story?", he asked, exasperated. The article that the news story references was just published in Nature, wh

"Did you read the original article or just the news story?", he asked, exasperated. The article that the news story references was just published in Nature, which has been around since 1869, is peered reviewed, well-respected, oft-cited, and has many distinguished moments, including the initial publication of Watson and Crick's work on DNA. "pseudoscience" might be unfair. Nice object lesson in the "social structures for doing something just aren’t there" though.

temphn 14y ago
When publishing a paper, "peer review" typically means that two or three scientists ask you for one round of revisions before publishing. It does not mean complete replication of the source code and data analysis. It means a cursory pass, not a complete unit test suite. Nature and Science in particular have been forced to retract many papers recently for failure to replicate. A short list of recent retracted Nature and Science papers: http://retractionwatch.wordpress.com/2010/10/14/highly-cited-harvard-stem-cell-scientist-retracts-nature-paper/ http://retractionwatch.wordpress.com/2010/10/14/highly-cited... http://www.genomesunzipped.org/2012/04/guest-post-accurate-identification-of-rna-editing-sites-from-high-throughput-sequencing-data.php http://www.genomesunzipped.org/2012/04/guest-post-accurate-i... http://www.scientificamerican.com/article.cfm?id=study-fails-to-confirm-existence http://www.scientificamerican.com/article.cfm?id=study-fails... And here is a recent climate paper by Gergis which was retracted after the blogosphere found flaws while attempting to replicate their work: http://climateaudit.org/2012/06/10/more-on-screening-in-gergis-et-al-2012/ http://climateaudit.org/2012/06/10/more-on-screening-in-gerg... All of those papers were beaten up after they were accepted because other scientists couldn't replicate their results. Perhaps we should see whether this too holds up to scrutiny for a few years before remaking the $100T world economy.
- scarmig 14y ago
  "retracted" is a bit strong: more accurate would be, put on hold temporarily while a statistical flaw is fixed. Even the author of the blog concedes that the overall paper could stand perfectly fine after the flaw is corrected. It's also interesting to note how the "auditors" compare to the scientists: while the scientists graciously send a thank you for pointing out a flaw and get to work on fixing it, the auditors seem almost entirely focused on crowing, getting a scalp and spreading conspiracy theories and innuendo. Interesting.
  - temphn 14y ago
    Well, the "auditors" are called denialists and attacked from many sides -- even though they are the only ones who publish full source code, push for raw data release, and actually try to replicate results. We are making $100T decisions on the basis of $1M papers and closed datasets. Regarding retraction, I think this is pretty unambiguous http://climateaudit.org/2012/06/08/gergis-et-al-put-on-hold/ http://climateaudit.org/2012/06/08/gergis-et-al-put-on-hold/ Dear Stephen, I am contacting you on behalf of all the authors of the Gergis et al (2012) study ‘Evidence of unusual late 20th century warming from an Australasian temperature reconstruction spanning the last millennium’ An issue has been identified in the processing of the data used in the study, which may affect the results. While the paper states that “both proxy climate and instrumental data were linearly detrended over the 1921–1990 period”, we discovered on Tuesday 5 June that the records used in the final analysis were not detrended for proxy selection, making this statement incorrect. Although this is an unfortunate data processing issue, it is likely to have implications for the results reported in the study. The journal has been contacted and the publication of the study has been put on hold. This is a normal part of science. The testing of scientific studies through independent analysis of data and methods strengthens the conclusions. In this study, an issue has been identified and the results are being re-checked. We would be grateful if you would post the notice below on your ClimateAudit web site. We would like to thank you and the participants at the ClimateAudit blog for your scrutiny of our study, which also identified this data processing issue. Thanks, David Karoly Print publication of scientific study put on hold An issue has been identified in the processing of the data used in the study, “Evidence of unusual late 20th century warming from an Australasian temperature reconstruction spanning the last millennium” by Joelle Gergis, Raphael Neukom, Stephen Phipps, Ailie Gallant and David Karoly, accepted for publication in the Journal of Climate. We are currently reviewing the data and results. Now, props to Karoly for being gracious in this takedown. Part of the problem is that a scientific paper is like a monolithic binary where a bugfix means going out and recalling every copy of the software out in the field. So paper authors usually are very resistant to bugfixes, as they are career ending.In this case Karoly severely damaged Gergis' career and may have hurt his own. From a purely self-interested perspective he should have just stonewalled the "denialists" like Michael Mann has. But he was honest as a scientist. Much better to move towards an open source culture where a "paper" is just the documentation for a major version number associated with a data/code repository.
- mkramlich 14y ago
  > And here is a recent climate paper by Gergis which was retracted after the blogosphere found flaws while attempting to replicate their work: > http://climateaudit.org/2012/06/10/more-on-screening-in-gerg.. http://climateaudit.org/2012/06/10/more-on-screening-in-gerg.... The link is to a website that clearly has the agenda of "anti-climate-crisis" and is run by a guy named Steve McIntyre, who, surprise surprise, appears to be from the mining and minerals exploration business, either as a finance guy or lawyer or both, per his own words here: "Previously, I spent about 35 years in the mining and mineral exploration business. During the last 20 years of this, I worked in the micro-cap exploration business and have a great deal of practical experience in dealing with prospectus and securities issues from the company point of view. Concepts like audit trails, due diligence and full, true and plain disclosure become second nature when you work in such an environment. ............ For McIntyre, undertaking this project has required an unpaid leave of absence from his career in mineral exploration financing, at the cost of over a year’s foregone earnings so far." ............. Wikipedia: "Stephen McIntyre is a Canadian mathematician, former minerals prospector, and semi-retired mining consultant (...) In 2011 he became Chairman of the Board of Trelawney Mining and Exploration Inc." Thus, probably a shill, and/or somebody with a financial incentive to hold his views, combined with a lack of relevant expertise. Bad combo. And unlikely to be trustable or objective. I'll stick with the views of the vast majority of climate scientists, thank you very much. However imperfect and human they may be. > Perhaps we should see whether this too holds up to scrutiny for a few years before remaking the $100T world economy. That line is not a strong argument, in my judgement, particularly the last 5 words. To get real specific, if the implication is that: (1) the total economy is $100T therefore we can't possibly change anything ... then, that argument is invalidated by the case that if we DON'T do anything, we may hurt or shrink that economy even more/worse than if we do take steps to address climate change. "We can't possibly make an incision in your skin, that could cause bleeding! Uh, there's a bullet lodged in your lung, you'll be much worse off if we DON'T remove it." (2) a $100T world economy means if we were to add say a 10% overhead cost to the production of all those goods services, the economy would somehow collapse and disappear, with millions or billions laid off, dogs and cats living together, oh my! No, that's not how it would work. First off, costs always get passed on. If people still needed a given thing, they'd still get it. Maybe they get less of one thing and more of another. And any money spent to reduce pollution or recapture carbon, etc. is money STILL spent on goods & services, on jobs, etc., so it still goes into people's hands, to in turn be spent again on other things. It might be the case that the billionaires and multi-millionaires who make profits from the existing state of the fossil fuel industry might make less profits -- or, they might make the same, if they shifted what they produce or reduced costs in other ways. For example, perhaps the oil & coal industry becomes less profitable. But the solar & wind industries grow and become more profitable. On the net, on the whole, big picture, that could be break even or even overall much better for the world economy. And there's no technical reason why someone like, say, a Koch, who gets a large amount of his current wealth from the oil industry, couldn't also invest in renewables, or in climate change mitigation technologies, and end up just shifting his profit sources over to that. The filthy rich can stay filthy rich. The blue collar guy driving trucks or doing trainable equipment maintenance in the natural gas industry can instead do it in the solar/wind/tidal/geothermal industry, etc. (3) remaking the $100T world economy is bad. Changes to our routines are bad! We shouldn't do that. ..... This is an invalid argument because the world economy is already changing. It's always in a constant state of continual change, anyway. Period. If anything, it's clearly not in a steady state, it's very chaotic, and the total nominal dollar size of the world economy just keeps growing and growing each year, seemingly without bound, just as the human population is growing. So it's already in a state of constant change, it's (roughly) growing without bound, and it already experiences retractions and booms and shocks. The climate science argument is, we might need to make certain changes in certain parts of our economy (not everything, just a subset of it), in order to prevent these other undesired things from happening, which could have a much bigger negative effect financially, and non-financially (quality of life terms, etc.) (4) remaking $100T world economy is bad, we shouldn't remake it! Again invalid because the climate change case is not to completely remake it. It wouldn't need to change 100% into something radically different. Just change enough to (a) take us off a collision course, and (b) ideally shift the total human impact on the climate & ecosystems into something sustainable and balanced. It could be a gradual partial change. But the longer we wait, the bigger the changes might have to be, all at once, and that's our fault and the fault of the folks who keep fighting it or dragging their heels. Never exercise and then ignore those weird sensations in your chest and suddenly one day you'll be making a radical change to your own personal economy in the form of a hospital emergency trip. Not picking on your personally, just this line of argument which I've seen made many times before on this issue.
InclinedPlane 14y ago
Yes, I read the original, in its entirety. As to peer-review, this was not published as either an "article" or "letter" (see: http://www.nature.com/nature/authors/gta/2a_Manuscript_formatting.pdf http://www.nature.com/nature/authors/gta/2a_Manuscript_forma...) but as a "review", which is more or less just a published essay. As such it falls under a much different peer review guideline. Even so, peer review is no surety of fact, far from it, it's nothing more than a signoff that the authors have seemingly followed the basic forms of reproducible scientific investigations. However, in this case I think that Nature has let down its readers even in publishing this essay as a "review". There is nothing that is not vague and generic in this essay. There is no model here, no theory. And no predictions nor falsifiability. It is not science in any sense and should not be treated as science. At best it is the shadowy outline of a future theory, at worst it is politics, mysticism, and superstition masquerading as science. And yes, reprinted in one of the most prestigious science journals of our times. That doesn't make it better, that makes it worse.

temphn 14y ago

When publishing a paper, "peer review" typically means that two or three scientists ask you for one round of revisions before publishing. It does not mean complete replication of the source code and data analysis. It means a cursory pass, not a complete unit test suite. Nature and Science in particular have been forced to retract many papers recently for failure to replicate. A short list of recent retracted Nature and Science papers: http://retractionwatch.wordpress.com/2010/10/14/highly-cited-harvard-stem-cell-scientist-retracts-nature-paper/ http://retractionwatch.wordpress.com/2010/10/14/highly-cited... http://www.genomesunzipped.org/2012/04/guest-post-accurate-identification-of-rna-editing-sites-from-high-throughput-sequencing-data.php http://www.genomesunzipped.org/2012/04/guest-post-accurate-i... http://www.scientificamerican.com/article.cfm?id=study-fails-to-confirm-existence http://www.scientificamerican.com/article.cfm?id=study-fails... And here is a recent climate paper by Gergis which was retracted after the blogosphere found flaws while attempting to replicate their work: http://climateaudit.org/2012/06/10/more-on-screening-in-gergis-et-al-2012/ http://climateaudit.org/2012/06/10/more-on-screening-in-gerg... All of those papers were beaten up after they were accepted because other scientists couldn't replicate their results. Perhaps we should see whether this too holds up to scrutiny for a few years before remaking the $100T world economy.

scarmig 14y ago
"retracted" is a bit strong: more accurate would be, put on hold temporarily while a statistical flaw is fixed. Even the author of the blog concedes that the overall paper could stand perfectly fine after the flaw is corrected. It's also interesting to note how the "auditors" compare to the scientists: while the scientists graciously send a thank you for pointing out a flaw and get to work on fixing it, the auditors seem almost entirely focused on crowing, getting a scalp and spreading conspiracy theories and innuendo. Interesting.
- temphn 14y ago
  Well, the "auditors" are called denialists and attacked from many sides -- even though they are the only ones who publish full source code, push for raw data release, and actually try to replicate results. We are making $100T decisions on the basis of $1M papers and closed datasets. Regarding retraction, I think this is pretty unambiguous http://climateaudit.org/2012/06/08/gergis-et-al-put-on-hold/ http://climateaudit.org/2012/06/08/gergis-et-al-put-on-hold/ Dear Stephen, I am contacting you on behalf of all the authors of the Gergis et al (2012) study ‘Evidence of unusual late 20th century warming from an Australasian temperature reconstruction spanning the last millennium’ An issue has been identified in the processing of the data used in the study, which may affect the results. While the paper states that “both proxy climate and instrumental data were linearly detrended over the 1921–1990 period”, we discovered on Tuesday 5 June that the records used in the final analysis were not detrended for proxy selection, making this statement incorrect. Although this is an unfortunate data processing issue, it is likely to have implications for the results reported in the study. The journal has been contacted and the publication of the study has been put on hold. This is a normal part of science. The testing of scientific studies through independent analysis of data and methods strengthens the conclusions. In this study, an issue has been identified and the results are being re-checked. We would be grateful if you would post the notice below on your ClimateAudit web site. We would like to thank you and the participants at the ClimateAudit blog for your scrutiny of our study, which also identified this data processing issue. Thanks, David Karoly Print publication of scientific study put on hold An issue has been identified in the processing of the data used in the study, “Evidence of unusual late 20th century warming from an Australasian temperature reconstruction spanning the last millennium” by Joelle Gergis, Raphael Neukom, Stephen Phipps, Ailie Gallant and David Karoly, accepted for publication in the Journal of Climate. We are currently reviewing the data and results. Now, props to Karoly for being gracious in this takedown. Part of the problem is that a scientific paper is like a monolithic binary where a bugfix means going out and recalling every copy of the software out in the field. So paper authors usually are very resistant to bugfixes, as they are career ending.In this case Karoly severely damaged Gergis' career and may have hurt his own. From a purely self-interested perspective he should have just stonewalled the "denialists" like Michael Mann has. But he was honest as a scientist. Much better to move towards an open source culture where a "paper" is just the documentation for a major version number associated with a data/code repository.
mkramlich 14y ago
> And here is a recent climate paper by Gergis which was retracted after the blogosphere found flaws while attempting to replicate their work: > http://climateaudit.org/2012/06/10/more-on-screening-in-gerg.. http://climateaudit.org/2012/06/10/more-on-screening-in-gerg.... The link is to a website that clearly has the agenda of "anti-climate-crisis" and is run by a guy named Steve McIntyre, who, surprise surprise, appears to be from the mining and minerals exploration business, either as a finance guy or lawyer or both, per his own words here: "Previously, I spent about 35 years in the mining and mineral exploration business. During the last 20 years of this, I worked in the micro-cap exploration business and have a great deal of practical experience in dealing with prospectus and securities issues from the company point of view. Concepts like audit trails, due diligence and full, true and plain disclosure become second nature when you work in such an environment. ............ For McIntyre, undertaking this project has required an unpaid leave of absence from his career in mineral exploration financing, at the cost of over a year’s foregone earnings so far." ............. Wikipedia: "Stephen McIntyre is a Canadian mathematician, former minerals prospector, and semi-retired mining consultant (...) In 2011 he became Chairman of the Board of Trelawney Mining and Exploration Inc." Thus, probably a shill, and/or somebody with a financial incentive to hold his views, combined with a lack of relevant expertise. Bad combo. And unlikely to be trustable or objective. I'll stick with the views of the vast majority of climate scientists, thank you very much. However imperfect and human they may be. > Perhaps we should see whether this too holds up to scrutiny for a few years before remaking the $100T world economy. That line is not a strong argument, in my judgement, particularly the last 5 words. To get real specific, if the implication is that: (1) the total economy is $100T therefore we can't possibly change anything ... then, that argument is invalidated by the case that if we DON'T do anything, we may hurt or shrink that economy even more/worse than if we do take steps to address climate change. "We can't possibly make an incision in your skin, that could cause bleeding! Uh, there's a bullet lodged in your lung, you'll be much worse off if we DON'T remove it." (2) a $100T world economy means if we were to add say a 10% overhead cost to the production of all those goods services, the economy would somehow collapse and disappear, with millions or billions laid off, dogs and cats living together, oh my! No, that's not how it would work. First off, costs always get passed on. If people still needed a given thing, they'd still get it. Maybe they get less of one thing and more of another. And any money spent to reduce pollution or recapture carbon, etc. is money STILL spent on goods & services, on jobs, etc., so it still goes into people's hands, to in turn be spent again on other things. It might be the case that the billionaires and multi-millionaires who make profits from the existing state of the fossil fuel industry might make less profits -- or, they might make the same, if they shifted what they produce or reduced costs in other ways. For example, perhaps the oil & coal industry becomes less profitable. But the solar & wind industries grow and become more profitable. On the net, on the whole, big picture, that could be break even or even overall much better for the world economy. And there's no technical reason why someone like, say, a Koch, who gets a large amount of his current wealth from the oil industry, couldn't also invest in renewables, or in climate change mitigation technologies, and end up just shifting his profit sources over to that. The filthy rich can stay filthy rich. The blue collar guy driving trucks or doing trainable equipment maintenance in the natural gas industry can instead do it in the solar/wind/tidal/geothermal industry, etc. (3) remaking the $100T world economy is bad. Changes to our routines are bad! We shouldn't do that. ..... This is an invalid argument because the world economy is already changing. It's always in a constant state of continual change, anyway. Period. If anything, it's clearly not in a steady state, it's very chaotic, and the total nominal dollar size of the world economy just keeps growing and growing each year, seemingly without bound, just as the human population is growing. So it's already in a state of constant change, it's (roughly) growing without bound, and it already experiences retractions and booms and shocks. The climate science argument is, we might need to make certain changes in certain parts of our economy (not everything, just a subset of it), in order to prevent these other undesired things from happening, which could have a much bigger negative effect financially, and non-financially (quality of life terms, etc.) (4) remaking $100T world economy is bad, we shouldn't remake it! Again invalid because the climate change case is not to completely remake it. It wouldn't need to change 100% into something radically different. Just change enough to (a) take us off a collision course, and (b) ideally shift the total human impact on the climate & ecosystems into something sustainable and balanced. It could be a gradual partial change. But the longer we wait, the bigger the changes might have to be, all at once, and that's our fault and the fault of the folks who keep fighting it or dragging their heels. Never exercise and then ignore those weird sensations in your chest and suddenly one day you'll be making a radical change to your own personal economy in the form of a hospital emergency trip. Not picking on your personally, just this line of argument which I've seen made many times before on this issue.

InclinedPlane 14y ago

Yes, I read the original, in its entirety. As to peer-review, this was not published as either an "article" or "letter" (see: http://www.nature.com/nature/authors/gta/2a_Manuscript_formatting.pdf http://www.nature.com/nature/authors/gta/2a_Manuscript_forma...) but as a "review", which is more or less just a published essay. As such it falls under a much different peer review guideline. Even so, peer review is no surety of fact, far from it, it's nothing more than a signoff that the authors have seemingly followed the basic forms of reproducible scientific investigations. However, in this case I think that Nature has let down its readers even in publishing this essay as a "review". There is nothing that is not vague and generic in this essay. There is no model here, no theory. And no predictions nor falsifiability. It is not science in any sense and should not be treated as science. At best it is the shadowy outline of a future theory, at worst it is politics, mysticism, and superstition masquerading as science. And yes, reprinted in one of the most prestigious science journals of our times. That doesn't make it better, that makes it worse.