3 ms·
Want to make sure you're familiar with the Pangeo community: www.pangeo.io I don't think any of these challenges are "solved", but there's a groundswell of tec
by counters 4y ago
Want to make sure you're familiar with the Pangeo community: www.pangeo.io
I don't think any of these challenges are "solved", but there's a groundswell of technology that is well-situated to make a big impact in these domains. The largest barriers that still remain are the ownership of engineering processes/workflow to transform larger gridded datasets to ARCO (analysis-ready, cloud-optimized) formats, as well as tooling to mediate between heterogeneous datasets (e.g. combining regular vs irregular or arbitrarily gridded data, such as land surveys or ZIP codes).
There are definitely players in the space working on these, but much is left to be done here.
- a_square_peg 4y ago+1 for Pangeo. We use these toolsets heavily (Xarray, Zarr, Dask) to run our service, which is essentially what you described as taking the larger gridded datasest to ARCO format. I think this is still a bit too heavy for casual Excel/GIS analysts so we try to make it as simple as possible for them to get climate data in CSV or NetCDF format for their work.