5 ms·

I don't think the title of the article matches the content. The GFS has not declined relative to the ECMWF. It just started from a lower skill level and its imp

I don't think the title of the article matches the content. The GFS has not declined relative to the ECMWF. It just started from a lower skill level and its improvements have not significantly closed the gap with ECMWF (as it has also improved). > Specifically, NOAA's global model, the UFS, is now in third or fourth place behind the European Center, the UK Meteorology Office, and often the Canadians. Would love to see evidence of that. It is well established that ECMWF is top in the game. I don't think it is reasonable to just state that Canada and UK are better without evidence. With that said, I agree the US should improve.

marcyb5st 2y ago
When it comes to medium range predictions. If you look at nowcasting, I believe the top dog is metnet 3 (https://arxiv.org/abs/2306.06079 https://arxiv.org/abs/2306.06079). On a tangential note, I worked on weather forecasting a bit before, and talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models. Not sure if they can't attract talent and so are somewhat stuck or if their higher ups are "old school" and can't really see the potential that this approach has to offer. It is just sad that the private sector is outclassing institutions and that these better models won't hit the public domain.
- plantain 2y ago
  > talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models This is trivially verifiable as not true. ECMWF is all over AI: https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-forecasting-system https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-... https://github.com/ecmwf-lab/ai-models https://github.com/ecmwf-lab/ai-models And they are hiring with competitive salaries (for Europe).
  - AStonesThrow 2y ago
    If only we'd put forecasting on the blonkchain so we could defend against 51% tornadoes
- carabiner 2y ago
  Windy.com offers ECMWF-AIFS models to paid subscribers. ECMWF seems to be all-in on the ML-based, as opposed to physical, NWP, but it's not clear ML approaches are superior.
- blueelephanttea 2y ago
  > When it comes to medium range predictions. If you look at nowcasting, I believe the top dog is metnet 3 (https://arxiv.org/abs/2306.06079 https://arxiv.org/abs/2306.06079). I don't consider 24 hour forecasts "medium range". And I don't really view MetNet-3 a competitor to GFS or ECMWF at all (currently).
plantain 2y ago
There is plenty of evidence - UKMO reliably outperforms GFS, as does DWD and many other centres. https://wmolcdnv.ecmwf.int/scores/mean/msl https://wmolcdnv.ecmwf.int/scores/mean/msl
- philipwhiuk 2y ago
  But the forecasting hasn't declined. It's better than it was.
- blueelephanttea 2y ago
  I would love to see a multi-year skill plot for all global models similar to what is commonly shared for GFS vs. ECMWF. Instead of having to parse a bunch of individual month comparisons for specific observations. Regardless, my criticism is towards the evidence presented in the article and the framing of the article.
  - lispisok 2y ago
    Like this? https://emc.ncep.noaa.gov/users/verification/global/gfs/ops/annual_long_term/ https://emc.ncep.noaa.gov/users/verification/global/gfs/ops/...

marcyb5st 2y ago

When it comes to medium range predictions. If you look at nowcasting, I believe the top dog is metnet 3 (https://arxiv.org/abs/2306.06079 https://arxiv.org/abs/2306.06079). On a tangential note, I worked on weather forecasting a bit before, and talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models. Not sure if they can't attract talent and so are somewhat stuck or if their higher ups are "old school" and can't really see the potential that this approach has to offer. It is just sad that the private sector is outclassing institutions and that these better models won't hit the public domain.

plantain 2y ago
> talking with some people still in the field it seems that these organizations (NOAA, ECMWF) are stuck with NWP and refuse to embrace AI models This is trivially verifiable as not true. ECMWF is all over AI: https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-forecasting-system https://www.ecmwf.int/en/newsletter/178/news/aifs-new-ecmwf-... https://github.com/ecmwf-lab/ai-models https://github.com/ecmwf-lab/ai-models And they are hiring with competitive salaries (for Europe).
- AStonesThrow 2y ago
  If only we'd put forecasting on the blonkchain so we could defend against 51% tornadoes
carabiner 2y ago
Windy.com offers ECMWF-AIFS models to paid subscribers. ECMWF seems to be all-in on the ML-based, as opposed to physical, NWP, but it's not clear ML approaches are superior.
blueelephanttea 2y ago
> When it comes to medium range predictions. If you look at nowcasting, I believe the top dog is metnet 3 (https://arxiv.org/abs/2306.06079 https://arxiv.org/abs/2306.06079). I don't consider 24 hour forecasts "medium range". And I don't really view MetNet-3 a competitor to GFS or ECMWF at all (currently).

plantain 2y ago

There is plenty of evidence - UKMO reliably outperforms GFS, as does DWD and many other centres. https://wmolcdnv.ecmwf.int/scores/mean/msl https://wmolcdnv.ecmwf.int/scores/mean/msl

philipwhiuk 2y ago
But the forecasting hasn't declined. It's better than it was.
blueelephanttea 2y ago
I would love to see a multi-year skill plot for all global models similar to what is commonly shared for GFS vs. ECMWF. Instead of having to parse a bunch of individual month comparisons for specific observations. Regardless, my criticism is towards the evidence presented in the article and the framing of the article.
- lispisok 2y ago
  Like this? https://emc.ncep.noaa.gov/users/verification/global/gfs/ops/annual_long_term/ https://emc.ncep.noaa.gov/users/verification/global/gfs/ops/...