4 ms·
There was a post last week about a weather API, so I thought I'd share mine as we had some major updates. This is a historical reanalysis data (ERA5 from ECMWF
by a_square_peg 5y ago
There was a post last week about a weather API, so I thought I'd share mine as we had some major updates.
This is a historical reanalysis data (ERA5 from ECMWF) post-processed for fast time-series access. The primary dataset is a gridded (0.25 x 0.25 lat/lon) dataset covering the entire world so the data can be extracted for any arbitrary location. Later this year, a higher resolution dataset (0.1 x 0.1 lat/lon, about 9km x 9km) will also be added.
I've been working on this because I believe having access to historical weather is important for us to understand how climate is changing where we live. I'm not a programmer by training so it's a little rough around the edges but I believe it's the largest repository of weather data outside the primary sources, with 50+ weather parameters covering 70+ years with hourly resolution. It should be about 10~100x faster than getting the data directly from the source, and without having to deal with NetCDF/GRIB formats.
All in all, it's about 100~200TB of data that's been post-processed/chunked to Zarr format, with about 1TB/day on-going. Current users are mostly analysts who occasionally need large volume of data and don't really have time to go to the primary sources. We've just rolled out a feature to request multiple locations also request regional data (returns the data in NetCDF format) although most users seem to be happy with querying for single locations at a time.
Any comments would be greatly appreciated.
- JNRowe 5y agoThe post from last week is presumably about https://open-meteo.com https://open-meteo.com ¹, for others(like me) who want to check that one out too. Colour me genuinely shocked that this much weather data is available to the public, fantastic project! ¹ https://news.ycombinator.com/item?id=28499910 https://news.ycombinator.com/item?id=28499910
- a_square_peg 5y agoThanks! Yes, that's the post I was referring to. :) Most weather data what we encounter from apps are actually public data - they're just cumbersome to work with. There is so much there but not as widely used as they should be.
- flarg 5y agoDo you have forecasting data as well? Could we use that to figure out how accurate forecasting is from various agencies?
- a_square_peg 5y agoIf you mean historical forecast data, we do archive it but only for seven days at the time. There are quite a bit of information comparing different model performance that you can look up (i.e. https://apps.ecmwf.int/wmolcdnv/ https://apps.ecmwf.int/wmolcdnv/) on the other hand. In general, the European model (from ECMWF) is considered to ahead of other agencies' models in terms of performance.