3 ms·
Why can't they just train on historical data?
by pkulak 3y ago
Why can't they just train on historical data?
- xapata 3y agoWe don't have enough data. There's only one universe, and it's helpful to train on counter-factual events.
- _visgean 3y agoERA5 is based on historical data. See it for yourself https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysis-era5-single-levels?tab=overview https://cds.climate.copernicus.eu/cdsapp#!/dataset/reanalysi..., https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-v5 https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-... I don't using raw historical data would work for any data intensive model - afaik the data is patchy - there are spots where we don't have that many datapoints - e.g. middle of ocean... Also there are new satelites that are only available for the last x years and you want to be able to use these for the new models. So you need a re-analysis of what it would look like if you had that data 40 years ago... Also its very convinient dataset because many other models trained on it: https://github.com/google-research/weatherbench2 https://github.com/google-research/weatherbench2 so easy to do benchmarking..