3 ms·
WRT breaking down by year, you could save some download time that way. Lets say you express the predicted error in the data not as a percentage of temperature
by VLM 11y ago
WRT breaking down by year, you could save some download time that way.
Lets say you express the predicted error in the data not as a percentage of temperature but as a time along the trendline. That would imply beyond a certain estimated error horizon, it doesn't really matter which year you download as long as you get the decade vaguely correct, once the temperature error figure exceeds a couple years of travel along the trend lines.
By example lets say today, here, 2080 CE, is 35C +/- 2C (to make the math easy) but the trend line is 0.1C/yr (unrealistic, but make the math easy) then +/- 2C is the same thing as saying +/- 20 years along the trendline, so I don't really need the 2080 data set I can do "roughly as well" with any data set downloaded from 2070 to 2090.
Yeah yeah I know calculus and the derivative is likely not constant or even linear and this is a really simplified way to look at statistics, but the general plan holds as a way to cut back on downloaded data required. Based on REAL statistical analysis you could come up with a formula that says you'll only increase the error bars 10% wider at year X if you skip every Y years of data. Where I'm guessing Y might exceed a decade at the extreme future years of the run. That could save an enormous amount of bandwidth without significantly impacting a visualization or analysis.
"Someone else" has to download it all, run the stat analysis, then tell us all, obviously it doesn't scale for all of us to download it all, run our own stat analysis, then go back in time and only download every Y-th year at or beyond year X. So thanks in advance, "Someone else"!
- godspiral 11y agoIts frustrating that they are not publishing the data in such chunks. Its good that they make every year available, but for analysis/reporting, you could reasonably just care about years 2030, 2050, 2100. 1 year of data would mean 120GB files or so (assuming 100 years total) Breaking it down by lattitude would mean 800MB lattide-year average data sets. Breaking it down by longitude would be very helpful too, and not much reason to not make that breakdown in addition to the lattitude one, which means 5mb data chunks. Its almost as though they are being intentionally uncooperative by dumping 12TB. Hopefully, some group makes a web interface that allows downloads in these manageable chunk sizes.