5 ms·

Unfortunately, this link does not include the raw CRU data which is at the heart of the debate. It is a repost of old data, which you can figure out by reading

Unfortunately, this link does not include the raw CRU data which is at the heart of the debate. It is a repost of old data, which you can figure out by reading the comments section of this post (http://www.realclimate.org/index.php/archives/2009/11/wheres-the-data/ http://www.realclimate.org/index.php/archives/2009/11/wheres...). This is disappointing. The University of East Anglia keeps talking about needing "permission" from national metrological service partners before releasing the data, but that should have been obtained before the first publication appeared. You couldn't pull this secret data thing in genomics, for example. Overall this appears to be an attempt at damage control, to get the issue out of the news. Yet those other "5%" of stations are critical for establishing the global trend. They should not have published without them. http://www.edie.net/news/news_story.asp?id=17333&channel=0&title=University+to+publish+all+climate+data http://www.edie.net/news/news_story.asp?id=17333&channel... Quote: The university has confirmed it will make all the data accessible as soon as it us released from a range of non-publication agreements, publication will be carried out in collaboration with the Met Office Hadley Centre. "We are grateful for the necessary support of the Met Office in requesting the permissions for releasing the information but understand that responses may take several months and that some countries may refuse permission due to the economic value of the data," continued Professor Davies. "The remaining data, to be published when permissions are given, generally cover areas of the world where there are fewer data collection stations."

Joeboy 17y ago
Last I heard they'd abandoned the raw data at some point in the '80s during an office move. Which is obviously less than ideal with hindsight but doesn't strike me as suspicious, as some seem to be suggesting. http://www.timesonline.co.uk/tol/news/environment/article6936328.ece http://www.timesonline.co.uk/tol/news/environment/article693...
- InclinedPlane 17y ago
  Losing data is rather suspicious when your private conversations include many direct mentions of how you would prefer to delete data rather than give it up to your opponents.
  - Joeboy 17y ago
    The email you're referring to is fairly embarrassing, but not being able to supply some source data from ~25 years ago still doesn't exactly look like a smoking gun to me. Storage was much more expensive and unwieldy back then.
    - mnemonicsloth 17y ago
      I'll just reproduce patio11's comment here: http://news.ycombinator.com/item?id=966620 http://news.ycombinator.com/item?id=966620 ... 1107454306.txt Professor Jones: Just sent loads of station data to Scott. Make sure he documents everything better this time ! And don’t leave stuff lying around on ftp sites - you never know who is trawling them. The two MMs [Patrick notes: he is referring to McIntyre and McKitrick] have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone. ... "Fairly Embarassing" doesn't cut it. Jones announces his intention to delete data in response to FoI requests rather than hand it over. An FoI request was indeed made. The data are discovered to be missing and cannot be recovered. Losing your data is a career-ender in any other scientific field. When you lost the data is irrelevant. In the context of the other emails, which reveal collusion to control the peer review process, marginalize critics, and censor comments on the very website linked in the original post, it's very clear they were trying to manipulate the scientific process.
      - Joeboy 17y ago
        http://www.timesonline.co.uk/tol/news/environment/article6936328.ece http://www.timesonline.co.uk/tol/news/environment/article693... implies the loss happened long before Jones wrote that email, and not on his watch.
        mnemonicsloth 17y ago
        Right. He didn't lose the data himself. He just conspired with his fellow scientists for 20 years to frustrate all attempts to bring that fact to light. In light of the malfeasance documented in those emails -- packing review boards, intimidating journals, marginalizing critics, etc. -- we have to wonder why. You would think that a scientist who worked in an especially contentious field, with well-funded political enemies and huge amounts of money at stake, would have taken extraordinary measures to establish his bona fides. That would include making his work especially open and accessible, and being at pains to document every single step in his research. That is the scientific ideal, after all.
ramanujan 17y ago
By the way, also note that if permissions are truly being refused because of the "economic value of the data", then the fact that this data is unavailable is quite significant. No one would put up a fight over releasing insignificant data, so there must be some unique and critical information there. Hence this is not a fuss over nothing. (by the by, the thing that pisses me off so much about this as a scientist is that data hiding is a direct assault on reproducibility. I can understand a company keeping some secret sauce to themselves -- no one expects Google to open source GFS. But the whole point of putting out a paper is so that the community of scholars can build on it.)
- Joeboy 17y ago
  > if permissions are truly being refused because of the "economic value of the data", then the fact that this data is unavailable is quite significant. No one would put up a fight over releasing insignificant data Its significance is that the people who supply the data can continue charging for it and are able to continue collecting data without being bankrupted. (Edit: how are you supposed to quote text on Hacker news?)
  - ramanujan 17y ago
    My understanding is that these are national services, i.e. government run and publicly financed. This is only my understanding -- to the best of my knowledge CRU has not actually responded to inquiries for exactly which countries and services are involved (so even that is secret). In any case we should surely be able to pay for the data to be publicly accessible if it is just a question of money. To upper bound it, let's say this data is worth a billion dollars. That is still a lot less than estimated mitigation costs (and is likely a very liberal upper bound!). Whatever it costs, funds for a public database should be the first line item on any climate change bill, with NIH-style penalties for failure to deposit government funded research data.
    - Joeboy 17y ago
      I agree with you, although I suspect most of the people criticizing the CRU aren't going to respond well to the suggestion that an extra billion dollars is required for climate research.
alan-crowe 17y ago
So what was the Kyoto Climate Treaty all about? Governments met in 1997 to agree to economically damaging restrictions to combat global warming. Twelve years later it comes out that the piddling value in national metrological service data sets isn't part of the global sacrifice in the common good and attempts to check how things are coming along are stymied by problems in getting data sets that have been shared globally cleared for general publication. What we think about excuses depends on when they are offered. If the UEA was talking about needing "permission" from national meteorological service partners in 1996 the excuse would work. In 1998 it would raise eye-brows: surely governments aren't blocking co-operation on meteorology, they have just signed a treaty. Once governments have agreed to cooperate on climate change, this excuse works less and less well as the years go by. Eventually the excuse morphs into a skeptic's talking point. They can argue that Kyoto was just about politics and nothing to do with science, pointing to the fact that it didn't provide for permission for scientist to publish, and the years rolled by and the omission never got fixed. How long should we allow, after Kyoto, for meteorological data sharing agreements to be sorted out at the intergovermental level? 5 years? 10 years? 20 years?

Joeboy 17y ago

Last I heard they'd abandoned the raw data at some point in the '80s during an office move. Which is obviously less than ideal with hindsight but doesn't strike me as suspicious, as some seem to be suggesting. http://www.timesonline.co.uk/tol/news/environment/article6936328.ece http://www.timesonline.co.uk/tol/news/environment/article693...

InclinedPlane 17y ago
Losing data is rather suspicious when your private conversations include many direct mentions of how you would prefer to delete data rather than give it up to your opponents.
- Joeboy 17y ago
  The email you're referring to is fairly embarrassing, but not being able to supply some source data from ~25 years ago still doesn't exactly look like a smoking gun to me. Storage was much more expensive and unwieldy back then.
  - mnemonicsloth 17y ago
    I'll just reproduce patio11's comment here: http://news.ycombinator.com/item?id=966620 http://news.ycombinator.com/item?id=966620 ... 1107454306.txt Professor Jones: Just sent loads of station data to Scott. Make sure he documents everything better this time ! And don’t leave stuff lying around on ftp sites - you never know who is trawling them. The two MMs [Patrick notes: he is referring to McIntyre and McKitrick] have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone. ... "Fairly Embarassing" doesn't cut it. Jones announces his intention to delete data in response to FoI requests rather than hand it over. An FoI request was indeed made. The data are discovered to be missing and cannot be recovered. Losing your data is a career-ender in any other scientific field. When you lost the data is irrelevant. In the context of the other emails, which reveal collusion to control the peer review process, marginalize critics, and censor comments on the very website linked in the original post, it's very clear they were trying to manipulate the scientific process.
    - Joeboy 17y ago
      http://www.timesonline.co.uk/tol/news/environment/article6936328.ece http://www.timesonline.co.uk/tol/news/environment/article693... implies the loss happened long before Jones wrote that email, and not on his watch.
      - mnemonicsloth 17y ago
        Right. He didn't lose the data himself. He just conspired with his fellow scientists for 20 years to frustrate all attempts to bring that fact to light. In light of the malfeasance documented in those emails -- packing review boards, intimidating journals, marginalizing critics, etc. -- we have to wonder why. You would think that a scientist who worked in an especially contentious field, with well-funded political enemies and huge amounts of money at stake, would have taken extraordinary measures to establish his bona fides. That would include making his work especially open and accessible, and being at pains to document every single step in his research. That is the scientific ideal, after all.

ramanujan 17y ago

By the way, also note that if permissions are truly being refused because of the "economic value of the data", then the fact that this data is unavailable is quite significant. No one would put up a fight over releasing insignificant data, so there must be some unique and critical information there. Hence this is not a fuss over nothing. (by the by, the thing that pisses me off so much about this as a scientist is that data hiding is a direct assault on reproducibility. I can understand a company keeping some secret sauce to themselves -- no one expects Google to open source GFS. But the whole point of putting out a paper is so that the community of scholars can build on it.)

Joeboy 17y ago
> if permissions are truly being refused because of the "economic value of the data", then the fact that this data is unavailable is quite significant. No one would put up a fight over releasing insignificant data Its significance is that the people who supply the data can continue charging for it and are able to continue collecting data without being bankrupted. (Edit: how are you supposed to quote text on Hacker news?)
- ramanujan 17y ago
  My understanding is that these are national services, i.e. government run and publicly financed. This is only my understanding -- to the best of my knowledge CRU has not actually responded to inquiries for exactly which countries and services are involved (so even that is secret). In any case we should surely be able to pay for the data to be publicly accessible if it is just a question of money. To upper bound it, let's say this data is worth a billion dollars. That is still a lot less than estimated mitigation costs (and is likely a very liberal upper bound!). Whatever it costs, funds for a public database should be the first line item on any climate change bill, with NIH-style penalties for failure to deposit government funded research data.
  - Joeboy 17y ago
    I agree with you, although I suspect most of the people criticizing the CRU aren't going to respond well to the suggestion that an extra billion dollars is required for climate research.

alan-crowe 17y ago

So what was the Kyoto Climate Treaty all about? Governments met in 1997 to agree to economically damaging restrictions to combat global warming. Twelve years later it comes out that the piddling value in national metrological service data sets isn't part of the global sacrifice in the common good and attempts to check how things are coming along are stymied by problems in getting data sets that have been shared globally cleared for general publication. What we think about excuses depends on when they are offered. If the UEA was talking about needing "permission" from national meteorological service partners in 1996 the excuse would work. In 1998 it would raise eye-brows: surely governments aren't blocking co-operation on meteorology, they have just signed a treaty. Once governments have agreed to cooperate on climate change, this excuse works less and less well as the years go by. Eventually the excuse morphs into a skeptic's talking point. They can argue that Kyoto was just about politics and nothing to do with science, pointing to the fact that it didn't provide for permission for scientist to publish, and the years rolled by and the omission never got fixed. How long should we allow, after Kyoto, for meteorological data sharing agreements to be sorted out at the intergovermental level? 5 years? 10 years? 20 years?