3 ms·
If you can link to the raw data so that I can wget it I will take a look at it.
by tosseraccount 11y ago
If you can link to the raw data so that I can wget it I will take a look at it.
- gnaritas 11y agoYou aren't capable of interpreting it, as is no one individual which is rather the point you're missing.
- tosseraccount 11y agoI'm not sure I understand. Can you explain? Are you saying people can not understand this data?
- astrodust 11y agoYou know, why don't you help us out and wget some data from CERN and give them some insight into their ongoing LHC experiments. I'm sure they could use a nudge in the right direction, and I bet a little bit of Perl or Python is all they need to solve some deep mysteries. Oh, and you have 6,000TB of disk space to download the data from the ATLAS sensor, right? Once you're done that there's all kinds of data regarding cancer treatments you can crunch through. I bet that's a weekend of work at the outside.
- tosseraccount 11y agoYou're right. I couldn't process 6 petabytes in perl. I'd use C for that amount of data.
- astrodust 11y agoGreat, so you're capable of writing highly parallel cluster-scale code in C that does intensive precision numerical analysis? Most choose vectorized FORTRAN or a combination of C++ and CUDA, but hey, knock yourself out. CERN has a 3700 core supercomputer to crunch through this kind of data. You can rent that on Amazon for about $800 an hour, so I guess you're good to go. Sorry to be so harsh here. While there's always desirable amount of "constructive naivety" necessary to try the impossible, you need to recognize that there's considerable amounts of expertise required to process and analyze data of this complexity at scale. This is not like a movie where six minutes of furious typing can solve any problem.
- tosseraccount 11y agoI bench marked fortran vs. x86_64 SSE extensions in C and .... C's fine. I'd rather have local clusters than Amazon or Google "cloud" any day of the week. Spotting a methodology bias is not that hard. Why the heck would you need CUDA ? NVIDIA ??? C'mon man.
- astrodust 11y agoAre you simply trolling at this point? If you're so confident in your ability to process this sort of data, please, post your follow-up on HN.
- deleted 11y ago[deleted]
- mturmon 11y agoYou should just stop with this line of thinking. You have to understand the scientific context and the limits of the data to learn anything. (Source: I know several contributors to the IPCC AR5 report, and I try to have the proper respect for their expertise.)
- b_emery 11y agoHere's a good place to start: ftp://ftp.cdc.noaa.gov/Datasets/20thC_ReanV2c/Monthlies/gaussian/monolevel/air.sfc.mon.mean.nc It's netcdf global monthly mean air temps. Pick some grid points and plot time series. Have fun.
- tosseraccount 11y agoThanks, Ill take a look. I hate HDF format !!!!