4 ms·

Hey buddy, I use this resource[1] all the time (and they're actually housed into the building next to me here at work). It's the long-term climate records from

by cwal37 11y ago

Hey buddy, I use this resource[1] all the time (and they're actually housed into the building next to me here at work). It's the long-term climate records from monitoring stations across the country, the United States Historical Climatology Network, and you can get daily data going back over a hundred years in some cases. For example, I've used it to crank out random temperature histograms in the past[2], and even plotted some long-term averages to decide on temperature and precipitation ranges when going on camping trips. It's really useful if you're willing to work with the data a tiny bit. You're getting csv files, so it's not terrible to work with in any way via Python or R, or whichever data-friendly language/tool you want to use. [1] http://cdiac.ornl.gov/epubs/ndp/ushcn/ushcn_map_interface.html http://cdiac.ornl.gov/epubs/ndp/ushcn/ushcn_map_interface.ht... [2] http://btus.us/decadal-histograms-of-monthly-yosemite-temperatures/ http://btus.us/decadal-histograms-of-monthly-yosemite-temper...

Alex3917 11y ago
So apparently in order to use this I need to enter a NESDIS ID. When I Google that, the first result I get informs me that a NESDIS ID is "an 8-character identifier assigned by NESDIS that becomes the RAWS identification number for use with the GOES satellite." Clicking on 'glossary' in the breadcrumbs then fails to take me to an actual glossary. I realize that it's theoretically possible to get this working. But it's not easy. What I want is a way to enter a postal code and get a JSON response with the precipitation for the last week. Without having to convert to scientology, learn GeoDjango, license GIS software, read an 800 page manual on parsing a bytestream from a satellite, etc.
- cwal37 11y ago
  Um, maybe to use it in an commercial setting? I've literally no idea what you're talking about as I've never come across that before. I've always just pulled down the station stuff I need by hand (since you can get all the station's data at once), or by some lightly mechanized webscraping. Your comment sounded significantly more chill than what it seems like you might need. It wouldn't be terribly difficult to mechanize this and get all the data at once, but I thought you wanted to occasionally see how much it rained, not build a service out of this. Does this page[1] not contain what you need? [1] http://cdiac.ornl.gov/ftp/ushcn_daily/ http://cdiac.ornl.gov/ftp/ushcn_daily/
  - tcdent 11y ago
    *.txt.gz is not exactly a modern data format. I share the parent's sentiment; finding easy access to good weather data is hard. Do you want your side project to be an experience in learning how to parse obscure formats and coordinating obsolete dependencies, or HTTP GET some JSON and immediately do something with the data?
    - cwal37 11y ago
      Not modern doesn't have to mean difficult to parse, it depends on your personal outlook. You can also download individually customized csv files if you want, which my original link led to. csv and txt files seem eminently reasonable to me when dealing with a huge historical cache of data that's being put out by a public institution for free.
    - Eridrus 11y ago
      You're shitting me, right? They're literally giving you all of the data in a plaintext format with documentation, and you're still complaining? Sure, having everything handed to you on a silver platter is nice, but why should they do your project for you?
      - spiralpolitik 11y ago
        Imagine what is going to happen when they discover the entire financial system is built on copying CSV files around by FTP.
        kbenson 11y ago
        Don't forget the emails back and forth and manual fixes and resubmissions from either end when there's a problem causing the CSV to not parse!
    - mvf0 11y ago
      I could write a node server that converts all that fixed length ASCII data into JSON faster than you guys can complain.
rgovind 11y ago
Try this http://www.forecastwatch.com/ http://www.forecastwatch.com/ I once bought data from them for couple of hundred dollars.

by cwal37 11y ago

Alex3917 11y ago

So apparently in order to use this I need to enter a NESDIS ID. When I Google that, the first result I get informs me that a NESDIS ID is "an 8-character identifier assigned by NESDIS that becomes the RAWS identification number for use with the GOES satellite." Clicking on 'glossary' in the breadcrumbs then fails to take me to an actual glossary. I realize that it's theoretically possible to get this working. But it's not easy. What I want is a way to enter a postal code and get a JSON response with the precipitation for the last week. Without having to convert to scientology, learn GeoDjango, license GIS software, read an 800 page manual on parsing a bytestream from a satellite, etc.

cwal37 11y ago
Um, maybe to use it in an commercial setting? I've literally no idea what you're talking about as I've never come across that before. I've always just pulled down the station stuff I need by hand (since you can get all the station's data at once), or by some lightly mechanized webscraping. Your comment sounded significantly more chill than what it seems like you might need. It wouldn't be terribly difficult to mechanize this and get all the data at once, but I thought you wanted to occasionally see how much it rained, not build a service out of this. Does this page[1] not contain what you need? [1] http://cdiac.ornl.gov/ftp/ushcn_daily/ http://cdiac.ornl.gov/ftp/ushcn_daily/
- tcdent 11y ago
  *.txt.gz is not exactly a modern data format. I share the parent's sentiment; finding easy access to good weather data is hard. Do you want your side project to be an experience in learning how to parse obscure formats and coordinating obsolete dependencies, or HTTP GET some JSON and immediately do something with the data?
  - cwal37 11y ago
    Not modern doesn't have to mean difficult to parse, it depends on your personal outlook. You can also download individually customized csv files if you want, which my original link led to. csv and txt files seem eminently reasonable to me when dealing with a huge historical cache of data that's being put out by a public institution for free.
  - Eridrus 11y ago
    You're shitting me, right? They're literally giving you all of the data in a plaintext format with documentation, and you're still complaining? Sure, having everything handed to you on a silver platter is nice, but why should they do your project for you?
    - spiralpolitik 11y ago
      Imagine what is going to happen when they discover the entire financial system is built on copying CSV files around by FTP.
      - kbenson 11y ago
        Don't forget the emails back and forth and manual fixes and resubmissions from either end when there's a problem causing the CSV to not parse!
  - mvf0 11y ago
    I could write a node server that converts all that fixed length ASCII data into JSON faster than you guys can complain.

rgovind 11y ago

Try this http://www.forecastwatch.com/ http://www.forecastwatch.com/ I once bought data from them for couple of hundred dollars.