4 ms·

It's specifically referring to these forecast models having accuracy about on par with the state-of-the-science global numerical weather forecast models. "Accur

It's specifically referring to these forecast models having accuracy about on par with the state-of-the-science global numerical weather forecast models. "Accuracy" here specifically means esoteric metrics like the "500mb anomaly correlation coefficient" (basically a summary statistic that tells you how well the 3D atmosphere fields predicted by the model match what we observe later on). This entire class of global numerical weather forecast models has had more-or-less a monotonic increase in forecast accuracy over the past five decades. E.g., a 72 hour forecast from the current generation of these models has similar error statistics to a 24 hour forecast from its predecessors in the early 2000's. What's special about these AI forecast models is that they are significantly cheaper to run than the existing global numerical weather forecast models. Modern meteorology involves a great deal of statistical analysis to overcome chaotic uncertainty. One way we build these statistical analyses is to run dozens of forecasts with the same model, using slightly different initial conditions, to see how the forecasts diverge. But these ensembles are very under-disperse - a few dozen members just doesn't fully sample the uncertainty. Now, if you can run 1000x the number of ensemble members, a whole new world of possibility opens up. And that's before you consider just training the AI system to directly output a posterior distribution representing this uncertainty in the first place...

sorokod 3y ago
Thanks for the explanation but my question was not about the cost but about the accuracy. Or is it the case that because of the reduced cost there is an opportunity to improve accuracy because more scenarios can be executed?
- counters 3y ago
  The problem in this world of AI/weather is that "accuracy" is an extremely fuzzy concept. The leading pack of AI-NWP models (NVIDIA's FourCastNet, DeepMind's Graphcast, Huawei PanguWeather), when compared on an apples-to-apples basis with the leading pack of traditional NWP systems (NOAA GFS and ECMWF HRES) have similar accuracy metrics. But here, "accuracy" is an esoteric term that is far removed from how an end user would perceive how "good" any of these given models are at predicting tomorrow's afternoon high temperature at their house. In the world of meteorology, the way you build an "accurate" (e.g. "user-perceived accuracy") forecast is to consume the entire previous class of forecasts and apply statistics/ML to post-process them. The greater your ability to probe uncertain in the forecasts from these models - e.g. by running larger ensembles or tailoring the ones you run to try to quantify the uncertainty more explicitly - the better your opportunity to improve those 'accuracy" metrics. So yes - the opportunity here is running larger sets of tailored forecast simulations as a way to statistically optimize forecast accuracy.

sorokod 3y ago

Thanks for the explanation but my question was not about the cost but about the accuracy. Or is it the case that because of the reduced cost there is an opportunity to improve accuracy because more scenarios can be executed?

counters 3y ago
The problem in this world of AI/weather is that "accuracy" is an extremely fuzzy concept. The leading pack of AI-NWP models (NVIDIA's FourCastNet, DeepMind's Graphcast, Huawei PanguWeather), when compared on an apples-to-apples basis with the leading pack of traditional NWP systems (NOAA GFS and ECMWF HRES) have similar accuracy metrics. But here, "accuracy" is an esoteric term that is far removed from how an end user would perceive how "good" any of these given models are at predicting tomorrow's afternoon high temperature at their house. In the world of meteorology, the way you build an "accurate" (e.g. "user-perceived accuracy") forecast is to consume the entire previous class of forecasts and apply statistics/ML to post-process them. The greater your ability to probe uncertain in the forecasts from these models - e.g. by running larger ensembles or tailoring the ones you run to try to quantify the uncertainty more explicitly - the better your opportunity to improve those 'accuracy" metrics. So yes - the opportunity here is running larger sets of tailored forecast simulations as a way to statistically optimize forecast accuracy.