7 ms·
CRU's "very artificial correction for decline" is dead code
- icodemyownshit 17y ago"My sincere apologies, I'm not an IDL programmer and I jumped the gun."
- kscaldef 17y agoIt's worth pointing out, for those who may not have RTFA or RTFComments, that this statement was a retraction from a commenter who attacked the claim that the code in question was dead code.
- deleted 17y ago[deleted]
- deleted 17y ago[deleted]
- deleted 17y ago[deleted]
- cwan 17y agoFrom Eric Raymond (ESR, in the comments of the original post that JGC responds to) - http://esr.ibiblio.org/?p=1447#comment-243042 http://esr.ibiblio.org/?p=1447#comment-243042: "As other have repeatedly pointed out, that code was written to be used for some kind of presentation that was false. The fact that the deceptive parts are commented out now does not change that at all. It might get them off the hook if we knew — for certain — that it had never been shown to anyone who didn’t know beforehand how the data was cooked and why. But since these peiple have conveniently lost or destroyed primary datasets and evaded FOIA requests, they don’t deserve the benefit of that doubt. We already know there’s a pattern of evasion and probable cause for criminal conspiracy charges from their own words."
- Tichy 17y ago"that code was written to be used for some kind of presentation that was false. " In other words, he doesn't know a damn thing. I can't help but feel that the anti-warming crowd (which includes ESR if I remember correctly, he wrote some long rant about peak oil?) is trying to blow a tiny mite up to be an elephant. It's typical revisionist behavior, just like the creationists who find a small bone that is not where biologists have predicted it would be, and they think they have refuted Darwin. In any case, this all seems to be very ad hominem (revisionists trying to discredit scientists instead of providing data). Even if somewhere out there in the vast internet a false chart is floating around, it does not refute global warming. (Note: I don't claim that global warming is real - I don't know. I just don't like the style of this "mite-gate"). The way to resolve this seems to create some proper charts off proper data.
- cwan 17y agoI agree - the remedy is transparency and to understand how, where and when these models are used - especially if they're being used to form significant public policy as is the case here. It seems reasonable to believe that the code was used for some purpose - even if it was dummy data - but clearly questions remain that should be asked and answered. What makes it even more troubling is the attitude of engagement of these scientists as if they are above being questioned when it is now revealed their emails suggest doubt, evasion and even suppression of valid but differing studies. I've come to believe that AGW probably exists but am not married to the idea either - but nor am I convinced we need massive corrective policy versus adaptation (heck - I live in Canada after all, and after last winter we could use a little warming). That said, and I'm not saying you're doing this but many proponents of AGW are attempting to minimize the implications in the emails but their impact cannot and should not be denied: http://www.guardian.co.uk/environment/georgemonbiot/2009/nov/25/monbiot-climate-leak-crisis-response http://www.guardian.co.uk/environment/georgemonbiot/2009/nov... Also - given the lack of transparency and the emails that have now come to light, the researchers most certainly open themselves up to questions that include whether or not the underlying data has been manipulated/adjusted such that the coding adjustments are no longer necessary http://wattsupwiththat.com/2009/11/26/mcintyre-data-from-the-hide-the-decline/ http://wattsupwiththat.com/2009/11/26/mcintyre-data-from-the...
- thras 17y agoAnyone searching for outright fraud in in these emails is going to be disappointed. The problem is not fraud, but rather scientists who believe that they are correct, and that their job is now not science but public relations. That said, finding this was a big deal: http://camirror.wordpress.com/2009/11/26/new-the-deleted-data/ http://camirror.wordpress.com/2009/11/26/new-the-deleted-dat...
- spoondan 17y agoThe only thing that's a "big deal" about that is the laziness and mendacity of its author. The data he found was published in 1998 and 2000, a fact he mentions without irony while arguing that the data was "hidden". The linked article also claims that the NOAA "deleted" the post-1960 data from a data set (briffa2001jgr3.txt) put together for an article that doesn't use these data. In other words, what really happened, is that Briffa and his collaborators created a data set only containing the data they used, and this derived data set was archived by NOAA. That's hardly nefarious or even the least bit worrisome. Now to the heart of the issue: Why was the post-1960 not used after 1998/2000? Why do all of the e-mails talk about "hiding the decline" and the code talk about stopping in 1960 to "avoid the decline"? The answer, it turns out, is right in the very Briffa 1998 paper your link mentions (but fails to cite). It's title is Reduced sensitivity of recent tree growth to temperature at high northern latitudes and it's about how temperatures reconstructed using Briffa's MXD (maximum latewood density) data diverge from actual temperature readings after 1960. The implication of the word "reconstructed" is very important to understand: Briffa's MXD data don't show an actual decline in temperature. They show a decline in a particular measurement of tree-ring density that had, until 1960, been a good indicator of actual temperature. All that's going on here is that data that is known to be inaccurate is not being used. It's no big deal.
- thras 17y agoNo, you are misinformed. Further, your tone is unhelpful. This was the first time that the data was available. As McIntyre points out in the top of his post, the decline was discussed in the 1998 and 2001 Briffa papers. Of course, scientists could not actually evaluate this because the data was not released. But that's not the issue. The issue is, as McIntyre also points out, that the IPCC 2001 graph (and later graphs) simply truncated the climate reconstruction data in 1960 and pasted the (more accurate) direct temperature station measurement data onto the graph after 1960. While this was done it plain sight, it's still lousy scientific practice. Why? After all, shouldn't more accurate data be used? The problem is that comparison of temperature reconstructions with direct measurement tells you a lot about the accuracy of your reconstruction. In this case it tells you a lot about the inaccuracy of the reconstruction and the sorts of signals that are missed. I'm sorry, the paleoclimate reconstructions available in the literature are not convincing. They often use dodgy statistical techniques, mixing data gathering steps with data correlation steps to increase apparent agreement with the instrument record. There have been huge problems with releasing data and there are often bizarre choices made in proxy record selection. Without good paleoclimate reconstructions, it's hard to know how far outside (or inside) the historical standard deviation for temperature we are. I happen to think it's an important question to have answered. A trillion dollar question.
- efaith 17y agoThere is a very similar file briffa_sep98_e.pro where the similar code does not appear to be dead code (or, at least, it does not die as soon as the code in the file the blogger was looking at). Most importantly, the above-quoted comment, "very artificial correction for decline" is not found in the code that the blogger looked at. But it is found in the code that I am talking about.
- DanielBMarkham 17y agoYay! A story about politics, programming, and climate science, all wrapped into one! It's like Christmas has come early here on HN. And the story just keeps getting better and better, too. Lots of twists and turns. So now we have code comments which are incendiary put ahead of, as it turn out, dead code. It's as if we found IRS code that had pieces in it like "This part makes sure all environmentalists get audited" but then the next part never runs. So it looks awful, but it does nothing. So I guess that's the key for any kind of code analysis, right? What does it do? You run the code. Has anybody taken the raw input and ran the code to make sure this piece of code is currently representative of the code used at the time? No wait -- we can't do that, can we? Because the data was lost/destroyed, right? So we have some version of some abstract pieces of code with incriminating comments in them that aren't being ran. On one hand, if you're looking for some kind of solid proof of scientists gone wild, you're never going to find it -- no matter how many times you add up nothing and nothing you'll get nothing -- because there are data and reproducibility problems that we can never resolve So it's always going to be he-said, she-said. On the other hand, that's been the problem with this issue all along -- data, models, and reproducibility. This case just brings it all out into the glorious sunshine for programmers like us to view. Outstanding.
- deleted 17y ago[deleted]
- jgrahamc 17y agoYes, but as someone points out in the comments there's another file where it is used: http://di2.nu/foia/harris-tree/briffa_sep98_e.pro http://di2.nu/foia/harris-tree/briffa_sep98_e.pro
- waterlesscloud 17y agoI await the explanation of this little detail.
- spamizbad 17y agoSee spoondan's reply.
- deleted 17y ago[deleted]
- doulaimi 17y agohttp://www.tradekay.com http://www.tradekay.com We are wholesaler of Nike Jordan and Other Shoes. We are a professional exporting company . We supply many kinds of Shoes, such as Nike Shoes, Jordan 1-23, Air Jordan, AF1, DUNK, Air max series etc. Most of them are in stock and can be supplied surely on time. All these shoes are packed with original-boxes and cards. AND We do wholesale and retail! All are extremely CHEAP, please visit: http://www.tradekay.com/productlist.asp?id=s1 http://www.tradekay.com/productlist.asp?id=s1 (UGG) http://www.tradekay.com/productlist.asp?id=n506 http://www.tradekay.com/productlist.asp?id=n506 (JACKET) http://www.tradekay.com/productlist.asp?id=s3 http://www.tradekay.com/productlist.asp?id=s3 (NBA Jersey) http://www.tradekay.com/productlist.asp?id=s12 http://www.tradekay.com/productlist.asp?id=s12 (JORDAN SHOES) http://www.tradekay.com/productlist.asp?id=s70 http://www.tradekay.com/productlist.asp?id=s70 (TSHIRTS) http://www.tradekay.com/productlist.asp?id=s75 http://www.tradekay.com/productlist.asp?id=s75 (JEANS) http://www.tradekay.com/productlist.asp?id=s69 http://www.tradekay.com/productlist.asp?id=s69 (HANDBAGS) http://www.tradekay.com/productlist.asp?id=s57 http://www.tradekay.com/productlist.asp?id=s57 (sunglass)
- joecode 17y agoMeanwhile, the Arctic is melting... Maybe some of the climate change scientists got caught up in the politics of the situation and inappropriately exaggerated matters. But consider this: every single national academy of science in the world agrees the basic theory of global warming is correct. That's pretty overwhelming. So there are two possibilities: it's all a big hoax by scientists, or it's nearly certain, but an unwarranted level of doubt has been fueled by, well, the industries that want to keep burning fuel... So what, really, sounds more likely? Let's say there's only an 80% chance the theory is correct. If it's not, then we're screwed anyhow. If it is, we might just be able prevent a whole lot of trouble if we act soon. How certain do we have to be?
- Joeboy 17y agoDoes anybody know what, if any, published work this data has made it into?