9 ms·

If this thing is trained upon the existing models input + output, doesn't this just mean that the AI is a duplicate/aggregate of the existing model/models? E.g.

by pintxo 2y ago

If this thing is trained upon the existing models input + output, doesn't this just mean that the AI is a duplicate/aggregate of the existing model/models? E.g. the quality should be the same if trained on a single model, might be better when trained on multiple? > The researchers trained Aurora on more than a million hours of data from six weather and climate models. After training the model, the team tweaked it to predict pollution and weather globally. The model generates a ten-day global weather forecast alongside the air-pollution prediction. Edit: Added the quote from the article where they confirm the model is trained on existing legacy models.

giantg2 2y ago
Is it possible it formed new inferences based on the analysis of the pervious models and their accuracy or errors? That would be the interesting part, but I'm not sure that was the case.
pintxo 2y ago
The interesting conclusion from this seems to be that the AI then could only get better when the underlying models improve, right? So we'd still need to keep working on them anyway, or not?
vessenes 2y ago
The idea generally for these models is: get an architecture that can copy outputs of a very slow process, ablate to see what you can discard from the architecture so that it will run more efficiently, train the heck out of it, then deliver something that’s 10-10000x more efficient than original process. This is the same broad plan used for Alphafold — and for things like Stable Diffusion turbo. It’s rare that these efforts yield interesting new output; they’re usually made specifically for the space/time speedup. In this case, weather predictions that are crazy fast/cheaper to run are seen as a public good, and so sending around a ‘compressed’ version of the big simulations is great.
- jhrmnn 2y ago
  Indeed, it's along these lines: https://www.microsoft.com/en-us/research/blog/ai4science-to-empower-the-fifth-paradigm-of-scientific-discovery/ https://www.microsoft.com/en-us/research/blog/ai4science-to-...
smeej 2y ago
This is a total layman's question, but the "legacy" models don't seem to get the weather forecast right very often. It's a running joke everywhere I've ever lived that weather forecasting is one of the few jobs where you can be wrong up to half the time and still keep your job. If you're making outdoor plans, you don't cancel your indoor backup plans until the forecast is less than 2-3 days out. Why would training an AI on many of the models that get mediocre results somehow produce a more reliable output than the current natural intelligences are able to get from the same models?
- tokai 2y ago
  Weather forecasts are plenty precise, and improving all the time. Just turns out that anecdotal evaluation of their precision is unreliable.
  - smeej 2y ago
    Are experts evaluating "precision" differently from laypeople? Like, from a layman's perspective, getting the forecasted high and low within 5 degrees and accurately predicting if it's going to rain (and if so, when) are probably the factors we're evaluating. We don't really notice or care if something like the atmospheric pressure forecast is right or wrong. We care about whether we're well-prepared for the temperature and whether we get wet, and we know from constant experience that the forecast 14 days out can change by 10+ degrees (F) and/or more than 50% chance of rain by the time it becomes tomorrow's forecast, so from our perspective, the forecast is "usually wrong" when it's more than a couple days out. Are we really, en masse, that wrong about what seems to be constant and universal experience? Or is "precision" measured very differently by professionals than what laypeople mean when they talk about "whether the weather forecast is right or wrong"?
    - pintxo 2y ago
      Given that the weather models are working on grid sizes 5km and up [1], and we often only get a report for a rough location like a whole city, meaning the prediction needs to be valid for 10s of square km, while our own personal observation range is often just a couple hundred square meters, I am not surprised that my personal sampling of a small area does not match the averaged prediction for a much larger area. But, having said that, temperature predictions seem to be fairly ok up to a week out. Rain is something else, but since we got our dog, I don't care any longer so much. I need to go out anyway ;-) And then I just take a look at the actual rain-radar data for my location to check the actual situation and the outlook for the next hour or so. [1] "the gridboxes in weather and climate models have sides that are between 5 kilometers (3 mi) and 300 kilometers (200 mi) in length" https://en.wikipedia.org/wiki/Numerical_weather_prediction https://en.wikipedia.org/wiki/Numerical_weather_prediction

by pintxo 2y ago

giantg2 2y ago

Is it possible it formed new inferences based on the analysis of the pervious models and their accuracy or errors? That would be the interesting part, but I'm not sure that was the case.

pintxo 2y ago

The interesting conclusion from this seems to be that the AI then could only get better when the underlying models improve, right? So we'd still need to keep working on them anyway, or not?

vessenes 2y ago

The idea generally for these models is: get an architecture that can copy outputs of a very slow process, ablate to see what you can discard from the architecture so that it will run more efficiently, train the heck out of it, then deliver something that’s 10-10000x more efficient than original process. This is the same broad plan used for Alphafold — and for things like Stable Diffusion turbo. It’s rare that these efforts yield interesting new output; they’re usually made specifically for the space/time speedup. In this case, weather predictions that are crazy fast/cheaper to run are seen as a public good, and so sending around a ‘compressed’ version of the big simulations is great.

jhrmnn 2y ago
Indeed, it's along these lines: https://www.microsoft.com/en-us/research/blog/ai4science-to-empower-the-fifth-paradigm-of-scientific-discovery/ https://www.microsoft.com/en-us/research/blog/ai4science-to-...

smeej 2y ago

This is a total layman's question, but the "legacy" models don't seem to get the weather forecast right very often. It's a running joke everywhere I've ever lived that weather forecasting is one of the few jobs where you can be wrong up to half the time and still keep your job. If you're making outdoor plans, you don't cancel your indoor backup plans until the forecast is less than 2-3 days out. Why would training an AI on many of the models that get mediocre results somehow produce a more reliable output than the current natural intelligences are able to get from the same models?

tokai 2y ago
Weather forecasts are plenty precise, and improving all the time. Just turns out that anecdotal evaluation of their precision is unreliable.
- smeej 2y ago
  Are experts evaluating "precision" differently from laypeople? Like, from a layman's perspective, getting the forecasted high and low within 5 degrees and accurately predicting if it's going to rain (and if so, when) are probably the factors we're evaluating. We don't really notice or care if something like the atmospheric pressure forecast is right or wrong. We care about whether we're well-prepared for the temperature and whether we get wet, and we know from constant experience that the forecast 14 days out can change by 10+ degrees (F) and/or more than 50% chance of rain by the time it becomes tomorrow's forecast, so from our perspective, the forecast is "usually wrong" when it's more than a couple days out. Are we really, en masse, that wrong about what seems to be constant and universal experience? Or is "precision" measured very differently by professionals than what laypeople mean when they talk about "whether the weather forecast is right or wrong"?
  - pintxo 2y ago
    Given that the weather models are working on grid sizes 5km and up [1], and we often only get a report for a rough location like a whole city, meaning the prediction needs to be valid for 10s of square km, while our own personal observation range is often just a couple hundred square meters, I am not surprised that my personal sampling of a small area does not match the averaged prediction for a much larger area. But, having said that, temperature predictions seem to be fairly ok up to a week out. Rain is something else, but since we got our dog, I don't care any longer so much. I need to go out anyway ;-) And then I just take a look at the actual rain-radar data for my location to check the actual situation and the outlook for the next hour or so. [1] "the gridboxes in weather and climate models have sides that are between 5 kilometers (3 mi) and 300 kilometers (200 mi) in length" https://en.wikipedia.org/wiki/Numerical_weather_prediction https://en.wikipedia.org/wiki/Numerical_weather_prediction