6 ms·

Yeah, I'm sure you have the time, inclination and expertise to look over billions of data points and make your own conclusions.

tosseraccount 11y ago
If you can link to the raw data so that I can wget it I will take a look at it.
- gnaritas 11y ago
  You aren't capable of interpreting it, as is no one individual which is rather the point you're missing.
  - tosseraccount 11y ago
    I'm not sure I understand. Can you explain? Are you saying people can not understand this data?
    - astrodust 11y ago
      You know, why don't you help us out and wget some data from CERN and give them some insight into their ongoing LHC experiments. I'm sure they could use a nudge in the right direction, and I bet a little bit of Perl or Python is all they need to solve some deep mysteries. Oh, and you have 6,000TB of disk space to download the data from the ATLAS sensor, right? Once you're done that there's all kinds of data regarding cancer treatments you can crunch through. I bet that's a weekend of work at the outside.
      - tosseraccount 11y ago
        You're right. I couldn't process 6 petabytes in perl. I'd use C for that amount of data.
        astrodust 11y ago
        Great, so you're capable of writing highly parallel cluster-scale code in C that does intensive precision numerical analysis? Most choose vectorized FORTRAN or a combination of C++ and CUDA, but hey, knock yourself out. CERN has a 3700 core supercomputer to crunch through this kind of data. You can rent that on Amazon for about $800 an hour, so I guess you're good to go. Sorry to be so harsh here. While there's always desirable amount of "constructive naivety" necessary to try the impossible, you need to recognize that there's considerable amounts of expertise required to process and analyze data of this complexity at scale. This is not like a movie where six minutes of furious typing can solve any problem.
        tosseraccount 11y ago
        I bench marked fortran vs. x86_64 SSE extensions in C and .... C's fine. I'd rather have local clusters than Amazon or Google "cloud" any day of the week. Spotting a methodology bias is not that hard. Why the heck would you need CUDA ? NVIDIA ??? C'mon man.
        astrodust 11y ago
        Are you simply trolling at this point? If you're so confident in your ability to process this sort of data, please, post your follow-up on HN.
    - deleted 11y ago
      [deleted]
- mturmon 11y ago
  You should just stop with this line of thinking. You have to understand the scientific context and the limits of the data to learn anything. (Source: I know several contributors to the IPCC AR5 report, and I try to have the proper respect for their expertise.)
- b_emery 11y ago
  Here's a good place to start: ftp://ftp.cdc.noaa.gov/Datasets/20thC_ReanV2c/Monthlies/gaussian/monolevel/air.sfc.mon.mean.nc It's netcdf global monthly mean air temps. Pick some grid points and plot time series. Have fun.
  - tosseraccount 11y ago
    Thanks, Ill take a look. I hate HDF format !!!!
triangleman 11y ago
A layperson named Steve McIntyre did just that, after reading the IPCC's 3rd Assessment Report and seeing the prominently featured "Hockey Stick" based on a paper by Mann et al., and wondering, hmm... perhaps this study is reproducible? The rabbit hole that McIntyre and his colleagues have found themselves in over the last 10 years has, in my view, demonstrated the value of hacker-types taking an interest in climate science. See here for a paper detailing the point where McIntyre's descent into the rabbit hole began: http://www.uoguelph.ca/~rmckitri/research/McKitrick-hockeystick.pdf http://www.uoguelph.ca/~rmckitri/research/McKitrick-hockeyst...
- hackuser 11y ago
  IIRC, despite the criticsim, the 'hockey stick' study and its author were proven right. Lots of criticism doesn't make it wrong, especially in politicized debates.
  - triangleman 11y ago
    How do you mean? Did you read the paper I linked to? In what way were Mann et al. "proven" right?
    - janlin1999 11y ago
      If read in isolation, the paper seems pretty convincing that something is amiss. However, it looks like some other scientists have generally confirmed the findings from Mann et al.: https://en.wikipedia.org/wiki/Hockey_stick_controversy#Further_reconstructions https://en.wikipedia.org/wiki/Hockey_stick_controversy#Furth... I haven't read those other papers.
- EdSharkey 11y ago
  From your link: "Of crucial importance here: the data for the bottom panel of Figure 6 is from a folder called CENSORED on Mann’s FTP site. He did this very experiment himself and discovered that the PCs lose their hockeystick shape when the Graybill-Idso series are removed. In so doing he discovered that the hockey stick is not a global pattern, it is driven by a flawed group of US proxies that experts do not consider valid as climate indicators. But he did not disclose this fatal weakness of his results, and it only came to light because of Stephen McIntyre’s laborious efforts." Geez, that's so damning.

tosseraccount 11y ago

If you can link to the raw data so that I can wget it I will take a look at it.

gnaritas 11y ago
You aren't capable of interpreting it, as is no one individual which is rather the point you're missing.
- tosseraccount 11y ago
  I'm not sure I understand. Can you explain? Are you saying people can not understand this data?
  - astrodust 11y ago
    You know, why don't you help us out and wget some data from CERN and give them some insight into their ongoing LHC experiments. I'm sure they could use a nudge in the right direction, and I bet a little bit of Perl or Python is all they need to solve some deep mysteries. Oh, and you have 6,000TB of disk space to download the data from the ATLAS sensor, right? Once you're done that there's all kinds of data regarding cancer treatments you can crunch through. I bet that's a weekend of work at the outside.
    - tosseraccount 11y ago
      You're right. I couldn't process 6 petabytes in perl. I'd use C for that amount of data.
      - astrodust 11y ago
        Great, so you're capable of writing highly parallel cluster-scale code in C that does intensive precision numerical analysis? Most choose vectorized FORTRAN or a combination of C++ and CUDA, but hey, knock yourself out. CERN has a 3700 core supercomputer to crunch through this kind of data. You can rent that on Amazon for about $800 an hour, so I guess you're good to go. Sorry to be so harsh here. While there's always desirable amount of "constructive naivety" necessary to try the impossible, you need to recognize that there's considerable amounts of expertise required to process and analyze data of this complexity at scale. This is not like a movie where six minutes of furious typing can solve any problem.
        tosseraccount 11y ago
        I bench marked fortran vs. x86_64 SSE extensions in C and .... C's fine. I'd rather have local clusters than Amazon or Google "cloud" any day of the week. Spotting a methodology bias is not that hard. Why the heck would you need CUDA ? NVIDIA ??? C'mon man.
        astrodust 11y ago
        Are you simply trolling at this point? If you're so confident in your ability to process this sort of data, please, post your follow-up on HN.
  - deleted 11y ago
    [deleted]
mturmon 11y ago
You should just stop with this line of thinking. You have to understand the scientific context and the limits of the data to learn anything. (Source: I know several contributors to the IPCC AR5 report, and I try to have the proper respect for their expertise.)
b_emery 11y ago
Here's a good place to start: ftp://ftp.cdc.noaa.gov/Datasets/20thC_ReanV2c/Monthlies/gaussian/monolevel/air.sfc.mon.mean.nc It's netcdf global monthly mean air temps. Pick some grid points and plot time series. Have fun.
- tosseraccount 11y ago
  Thanks, Ill take a look. I hate HDF format !!!!

triangleman 11y ago

A layperson named Steve McIntyre did just that, after reading the IPCC's 3rd Assessment Report and seeing the prominently featured "Hockey Stick" based on a paper by Mann et al., and wondering, hmm... perhaps this study is reproducible? The rabbit hole that McIntyre and his colleagues have found themselves in over the last 10 years has, in my view, demonstrated the value of hacker-types taking an interest in climate science. See here for a paper detailing the point where McIntyre's descent into the rabbit hole began: http://www.uoguelph.ca/~rmckitri/research/McKitrick-hockeystick.pdf http://www.uoguelph.ca/~rmckitri/research/McKitrick-hockeyst...

hackuser 11y ago
IIRC, despite the criticsim, the 'hockey stick' study and its author were proven right. Lots of criticism doesn't make it wrong, especially in politicized debates.
- triangleman 11y ago
  How do you mean? Did you read the paper I linked to? In what way were Mann et al. "proven" right?
  - janlin1999 11y ago
    If read in isolation, the paper seems pretty convincing that something is amiss. However, it looks like some other scientists have generally confirmed the findings from Mann et al.: https://en.wikipedia.org/wiki/Hockey_stick_controversy#Further_reconstructions https://en.wikipedia.org/wiki/Hockey_stick_controversy#Furth... I haven't read those other papers.
EdSharkey 11y ago
From your link: "Of crucial importance here: the data for the bottom panel of Figure 6 is from a folder called CENSORED on Mann’s FTP site. He did this very experiment himself and discovered that the PCs lose their hockeystick shape when the Graybill-Idso series are removed. In so doing he discovered that the hockey stick is not a global pattern, it is driven by a flawed group of US proxies that experts do not consider valid as climate indicators. But he did not disclose this fatal weakness of his results, and it only came to light because of Stephen McIntyre’s laborious efforts." Geez, that's so damning.