4 ms·

I find this quite surprising actually. You'd think predicting the weather is mostly a matter of fast computation. The physical rules are well understood, so to

I find this quite surprising actually. You'd think predicting the weather is mostly a matter of fast computation. The physical rules are well understood, so to get a better estimate use a finer mesh in your finite element computation and use a smaller time scale in estimating your differential equations. Neural networks are notoriously bad at exact approximation. I mean you can never beat a calculator when the issue is doing calculations. So apparently the AI found some shortcut for doing the actual computational work. That is also surprising as weather is a chaotic system. Shortcuts should not exist. Long story short, I don't get what's going on here.

uoaei 3y ago
> The physical rules are well understood Nope. They're constantly updating these models with really finnicky things like cloud nucleation rates that differ depending on which tree species's pollen is in the air. They've gotten a lot better (~2 day to ~7 day hi-res forecasts) but they're still wrong a lot of the time. The reason is the chaos as you say, however, chaos is deterministic, so, that a deterministic method can approximate a deterministic system is really not the surprising part. You don't get what's going on here because your baseline understanding is a lot worse than you think it is. What they're doing is skipping literal numerical simulation in favor of graph- (attention-) based approaches. Typical weather models simulate pretty fine resolution and return hourly forecasts. Google's new approach is learning an approximate Markov model at 6 hours resolution directly so they don't need to run on massive supercomputers.
- flir 3y ago
  It's a model of a model? And it turns out to be better? That's so counter-intuitive I'm kinda amazed anyone even bothered to research it, let alone that it worked. Uh..... now do horse racing.
  - uoaei 3y ago
    "All models are wrong, some models are useful." Some are more wrong and more useful simultaneously ;) This is actually the typical state of things in numerical simulation: we have infinite-resolution differential equations modeling such physical systems, but to implement them in silico we need to discretize and approximate various aspects of those models to achieve usefulness re: time and accuracy. Google has merely gone one level further in the tradeoff. For more info on Google's approach, look into surrogate models. It's becoming more common especially in things like weather and geology.
karaterobot 3y ago
I used to think so too, but evidently weather forecasting is a much harder problem than it seems from the outside. I was talking to a physicist who told me who had first wanted to get into weather modeling, but that it was too hard. I think his quote was something like: "those guys are hard. core."
- boxed 3y ago
  My grandfather specifically chose meteorology as a field because it had the most numbers heh.
- SoftTalker 3y ago
  Too many variables. The combinatorial complexity exceeds what any computational model can deal with.
  - staunton 3y ago
    It's a chaotic system, one could equally well wonder how it's possible at all, even given insane amounts of compute, especially forcasting days and weeks ahead...
giovannibonetti 3y ago
AI/ML's bitter lesson [1] applies again. In this case, the AI model may have learned a more practical model than the one human researchers painstakingly came up with by applying piles and piles of physics research. [1] http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html
- savanaly 3y ago
  That’s only superficially similar to ai’s bitter lesson. The bitter lesson is about methods to achieve results in AI, not about comparing AI methods to non-AI methods.
- uoaei 3y ago
  Hardly. They trained on the output of numerical simulations, so it's basically a method for summarizing approximate dynamics of numerical simulations themselves.
- amelius 3y ago
  > by applying piles and piles of physics research. You mean by remembering piles and piles of example data and interpolating between it.
3cats-in-a-coat 3y ago
This is essentially the same exact problem as a classic chess playing program, recursively computing all possibilities N moves ahead, and an AI which "groks" the game's patterns and knows where to focus fewer resources with greater success. This translates especially well to games like Go, where computing all moves is not even pragmatically possible the classic way. But AI beats the best Go players. Raw models are excellent for establishing the theory, and for training the AI. But... the AI is better at figuring out more effective, precise, and efficient model within itself, based on both synthetic (based on models) and real data (actual weather patterns). EDIT: And just to point out, this is not just an AI phenomenon. You are a neural network. And "intuition" is the sense of predicting outcomes you develop, without knowing how and why precisely. This is why I frown upon people with academic knowledge who dismiss people with say engineering or other practical experience in a field. A farmer may not tell you why doing things a weird way results in amazing crop yields, but he gets the gains, and when theory doesn't correlate with reality, it's not reality that's wrong, but the theory. To recap, nothing beats "learning by example". And AI learns by example. Of course, the formal theoretic models that we can collectively share, explain, and evolve over time have their own strong benefits and have allowed us to grow as a civilization. Computers are in effect "formal computation machines". I don't think we'll run AI for long on digital circuits and it's a clumsy workaround. Computers will have analog processing units for AI and digital processing units for cold, hard logic and data. And the combination is the most powerful approach of all.
bannedbybros 3y ago
[dead]
sheepshear 3y ago
The accuracy improvement boils down to representing more salient features in the model. The humans got a head start figuring out what to model, but the machine figures it out faster, so it caught up and surpassed them. Now it models more important stuff. The speed difference is a side effect of completely different implementations. One is a step-by-step simulator, the other is an input/output pattern matcher.
deleted 3y ago
[deleted]
mnw21cam 3y ago
I did my doctorate in the Met Office. Weather forecasting is two separate problems. The first of these is physics - given the state of the atmosphere right now, what will it do. And this is hard, because there are so many different effects, combined with the fact that our computational models have a limited resolution. There's a huge amount of work that goes into making the simulation behave like a real atmosphere does, and a lot of that is faking what is going on at a smaller scale than the model grid. The second part is to work out what the current state of the atmosphere is. This is what takes vast amounts of computing power. We don't have an observation station at every grid point and at every altitude in the atmospheric model, so we need to find some other way to infer what the atmospheric state is from the observations that we can from it. Many of these observations are limited in locality, like weather stations, or are a complex function of the atmospheric state, like satellite imagery. The light reaching a satellite has been affected by all the layers of the atmosphere it passes through, and sometimes in a highly nonlinear way. In order to calculate the atmospheric state, we need to take the previous forecast of the current atmospheric state, compare it to the observations, then find the first derivative (as in calculus) of the observation function so that we can adjust the atmospheric state estimate to the new best estimate. This is then complicated by the fact that the observations were not all taken at a single time snapshot - for instance polar orbiting satellites will be taking observations spread out in time. So, we need to use the physics model to wind the atmospheric state back in time to when the observation was taken, find the first derivative of that too, and use it to reconcile the observations with the atmospheric state. It's a massive minimisation/optimisation problem with millions of free variables, and in some cases we need the second derivative of all these functions too in order to make the whole thing converge correctly and within a reasonable amount of time. It takes a reasonable number of iterations of the minimisation algorithm to get it settle on a solution. The problem is that these minimisation methods often assume that the function being minimised is reasonably linear, which certain atmospheric phenomena are not (such as clouds), so certain observations have to be left out of the analysis to avoid the whole thing blowing up. My doctorate was looking to see if the nonlinearity involved in a cloud forming as air was moving upwards could be used to translate a time-series of satellite infra-red observations into a measurement of vertical air velocity. The answer was that this single form of nonlinearity made the whole minimisation process fairly dire. I implemented a fairly simple not-quite-machine-learning approach, and it was able to find a solution that was almost as accurate but much more reliable than the traditional minimisation method. Also, to answer the dead sibling comment asking whether weather is really a chaotic system - yes it is. The definition of a chaotic system is that a small change in current state results in a very large change in outcome, and that's definitely the case. The improvements in weather forecasting over the last few decades have been due to improvements in solving both of the above problems - the physics has been pinned down better, but we're also better as working out the current atmospheric state fairly accurately, and that has added something like a day of forecasting accuracy each decade we have been working on it.
londons_explore 3y ago
Shortcuts 100% exist. Imagine another physical problem. Simulating a sand grain and how it bounces off other sand grains or lodges against them. If you wanted to simulate a sand mountain, you could use a massive amount of compute and predict the location and behaviour of every single grain. Or, you could take a bunch of well-known shortcuts and just know that sand sits in a heap at the angle-of-repose. That angle decides how steep the mountain will be. any steeper and it will tumble till it's at that angle. Suddenly, the computation is dramatically reduced, and you get pretty much the same result.
- tovej 3y ago
  The point was that weather, unlike a sandheap, is a chaotic hydrodynamic system with turbulent flows, that means it's computationally intractable to do exactly, which is why weather forecasts are only good for a few days anyway. The example you gave does not really explain anything.
  - londons_explore 3y ago
    The sandheap is chaotic too - just one sand grain tumbling can be enough to start a landslip. But the end result tends not to depend on the minute details - if sand grain A didn't cause the landslip, then a few seconds later sand grain B would have.
    - tovej 3y ago
      That's not chaos, the outcome is the same even if the input varies. Chaos is when the outcome differs greatly with small changes in input.
- twayt 3y ago
  You get the same result in a short span of time, heck you may even get a reliable error bound. Where this falls apart is that error accumulates over time and not just for one heap of sand but for many such heaps of sand that also interact with other heaps of sand. Predicting weather for the next hour is trivial. Aviation runs on the fact that you can forecast fairly accurately into the next hour most of the time. The difficulty scales superlinearly over time due to the error accumulation over predictions
empath-nirvana 3y ago
> Neural networks are notoriously bad at exact approximation. Neural networks can compute pretty much anything. There's no reason, given the same inputs and with enough trainining data that it shouldn't be able to discover the same physical laws that were hard-coded previously.
T-A 3y ago
https://www.science.org/content/article/models-galaxies-atoms-simple-ai-shortcuts-speed-simulations-billions-times https://www.science.org/content/article/models-galaxies-atom... https://towardsdatascience.com/physics-informed-neural-networks-pinns-an-intuitive-guide-fff138069563 https://towardsdatascience.com/physics-informed-neural-netwo... https://maziarraissi.github.io/PINNs/ https://maziarraissi.github.io/PINNs/ https://arxiv.org/abs/2001.08055 https://arxiv.org/abs/2001.08055 https://arxiv.org/abs/2009.11990 https://arxiv.org/abs/2009.11990
amelius 3y ago
It's more like table lookup or interpolation than actual computation.
cowboysauce 3y ago
> So apparently the AI found some shortcut for doing the actual computational work. That is also surprising as weather is a chaotic system. Shortcuts should not exist. Why do you say that shortcuts should not exist? Even very basic statements like "falling pressure and increasing humidity indicate a storm is coming" are generally valid. I've done a little bit of storm-chasing and I'm able to point out areas that are likely to experience severe thunderstorms based on a few values (CAPE, dew point, wind shear, etc). I'm sure forecast meteorologists have even better skills. Are those not shortcuts?

uoaei 3y ago

> The physical rules are well understood Nope. They're constantly updating these models with really finnicky things like cloud nucleation rates that differ depending on which tree species's pollen is in the air. They've gotten a lot better (~2 day to ~7 day hi-res forecasts) but they're still wrong a lot of the time. The reason is the chaos as you say, however, chaos is deterministic, so, that a deterministic method can approximate a deterministic system is really not the surprising part. You don't get what's going on here because your baseline understanding is a lot worse than you think it is. What they're doing is skipping literal numerical simulation in favor of graph- (attention-) based approaches. Typical weather models simulate pretty fine resolution and return hourly forecasts. Google's new approach is learning an approximate Markov model at 6 hours resolution directly so they don't need to run on massive supercomputers.

flir 3y ago
It's a model of a model? And it turns out to be better? That's so counter-intuitive I'm kinda amazed anyone even bothered to research it, let alone that it worked. Uh..... now do horse racing.
- uoaei 3y ago
  "All models are wrong, some models are useful." Some are more wrong and more useful simultaneously ;) This is actually the typical state of things in numerical simulation: we have infinite-resolution differential equations modeling such physical systems, but to implement them in silico we need to discretize and approximate various aspects of those models to achieve usefulness re: time and accuracy. Google has merely gone one level further in the tradeoff. For more info on Google's approach, look into surrogate models. It's becoming more common especially in things like weather and geology.

karaterobot 3y ago

I used to think so too, but evidently weather forecasting is a much harder problem than it seems from the outside. I was talking to a physicist who told me who had first wanted to get into weather modeling, but that it was too hard. I think his quote was something like: "those guys are hard. core."

boxed 3y ago
My grandfather specifically chose meteorology as a field because it had the most numbers heh.
SoftTalker 3y ago
Too many variables. The combinatorial complexity exceeds what any computational model can deal with.
- staunton 3y ago
  It's a chaotic system, one could equally well wonder how it's possible at all, even given insane amounts of compute, especially forcasting days and weeks ahead...

giovannibonetti 3y ago

AI/ML's bitter lesson [1] applies again. In this case, the AI model may have learned a more practical model than the one human researchers painstakingly came up with by applying piles and piles of physics research. [1] http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html

savanaly 3y ago
That’s only superficially similar to ai’s bitter lesson. The bitter lesson is about methods to achieve results in AI, not about comparing AI methods to non-AI methods.
uoaei 3y ago
Hardly. They trained on the output of numerical simulations, so it's basically a method for summarizing approximate dynamics of numerical simulations themselves.
amelius 3y ago
> by applying piles and piles of physics research. You mean by remembering piles and piles of example data and interpolating between it.

3cats-in-a-coat 3y ago

This is essentially the same exact problem as a classic chess playing program, recursively computing all possibilities N moves ahead, and an AI which "groks" the game's patterns and knows where to focus fewer resources with greater success. This translates especially well to games like Go, where computing all moves is not even pragmatically possible the classic way. But AI beats the best Go players. Raw models are excellent for establishing the theory, and for training the AI. But... the AI is better at figuring out more effective, precise, and efficient model within itself, based on both synthetic (based on models) and real data (actual weather patterns). EDIT: And just to point out, this is not just an AI phenomenon. You are a neural network. And "intuition" is the sense of predicting outcomes you develop, without knowing how and why precisely. This is why I frown upon people with academic knowledge who dismiss people with say engineering or other practical experience in a field. A farmer may not tell you why doing things a weird way results in amazing crop yields, but he gets the gains, and when theory doesn't correlate with reality, it's not reality that's wrong, but the theory. To recap, nothing beats "learning by example". And AI learns by example. Of course, the formal theoretic models that we can collectively share, explain, and evolve over time have their own strong benefits and have allowed us to grow as a civilization. Computers are in effect "formal computation machines". I don't think we'll run AI for long on digital circuits and it's a clumsy workaround. Computers will have analog processing units for AI and digital processing units for cold, hard logic and data. And the combination is the most powerful approach of all.

bannedbybros 3y ago

[dead]

sheepshear 3y ago

The accuracy improvement boils down to representing more salient features in the model. The humans got a head start figuring out what to model, but the machine figures it out faster, so it caught up and surpassed them. Now it models more important stuff. The speed difference is a side effect of completely different implementations. One is a step-by-step simulator, the other is an input/output pattern matcher.

deleted 3y ago

[deleted]

mnw21cam 3y ago

I did my doctorate in the Met Office. Weather forecasting is two separate problems. The first of these is physics - given the state of the atmosphere right now, what will it do. And this is hard, because there are so many different effects, combined with the fact that our computational models have a limited resolution. There's a huge amount of work that goes into making the simulation behave like a real atmosphere does, and a lot of that is faking what is going on at a smaller scale than the model grid. The second part is to work out what the current state of the atmosphere is. This is what takes vast amounts of computing power. We don't have an observation station at every grid point and at every altitude in the atmospheric model, so we need to find some other way to infer what the atmospheric state is from the observations that we can from it. Many of these observations are limited in locality, like weather stations, or are a complex function of the atmospheric state, like satellite imagery. The light reaching a satellite has been affected by all the layers of the atmosphere it passes through, and sometimes in a highly nonlinear way. In order to calculate the atmospheric state, we need to take the previous forecast of the current atmospheric state, compare it to the observations, then find the first derivative (as in calculus) of the observation function so that we can adjust the atmospheric state estimate to the new best estimate. This is then complicated by the fact that the observations were not all taken at a single time snapshot - for instance polar orbiting satellites will be taking observations spread out in time. So, we need to use the physics model to wind the atmospheric state back in time to when the observation was taken, find the first derivative of that too, and use it to reconcile the observations with the atmospheric state. It's a massive minimisation/optimisation problem with millions of free variables, and in some cases we need the second derivative of all these functions too in order to make the whole thing converge correctly and within a reasonable amount of time. It takes a reasonable number of iterations of the minimisation algorithm to get it settle on a solution. The problem is that these minimisation methods often assume that the function being minimised is reasonably linear, which certain atmospheric phenomena are not (such as clouds), so certain observations have to be left out of the analysis to avoid the whole thing blowing up. My doctorate was looking to see if the nonlinearity involved in a cloud forming as air was moving upwards could be used to translate a time-series of satellite infra-red observations into a measurement of vertical air velocity. The answer was that this single form of nonlinearity made the whole minimisation process fairly dire. I implemented a fairly simple not-quite-machine-learning approach, and it was able to find a solution that was almost as accurate but much more reliable than the traditional minimisation method. Also, to answer the dead sibling comment asking whether weather is really a chaotic system - yes it is. The definition of a chaotic system is that a small change in current state results in a very large change in outcome, and that's definitely the case. The improvements in weather forecasting over the last few decades have been due to improvements in solving both of the above problems - the physics has been pinned down better, but we're also better as working out the current atmospheric state fairly accurately, and that has added something like a day of forecasting accuracy each decade we have been working on it.

londons_explore 3y ago

Shortcuts 100% exist. Imagine another physical problem. Simulating a sand grain and how it bounces off other sand grains or lodges against them. If you wanted to simulate a sand mountain, you could use a massive amount of compute and predict the location and behaviour of every single grain. Or, you could take a bunch of well-known shortcuts and just know that sand sits in a heap at the angle-of-repose. That angle decides how steep the mountain will be. any steeper and it will tumble till it's at that angle. Suddenly, the computation is dramatically reduced, and you get pretty much the same result.

tovej 3y ago
The point was that weather, unlike a sandheap, is a chaotic hydrodynamic system with turbulent flows, that means it's computationally intractable to do exactly, which is why weather forecasts are only good for a few days anyway. The example you gave does not really explain anything.
- londons_explore 3y ago
  The sandheap is chaotic too - just one sand grain tumbling can be enough to start a landslip. But the end result tends not to depend on the minute details - if sand grain A didn't cause the landslip, then a few seconds later sand grain B would have.
  - tovej 3y ago
    That's not chaos, the outcome is the same even if the input varies. Chaos is when the outcome differs greatly with small changes in input.
twayt 3y ago
You get the same result in a short span of time, heck you may even get a reliable error bound. Where this falls apart is that error accumulates over time and not just for one heap of sand but for many such heaps of sand that also interact with other heaps of sand. Predicting weather for the next hour is trivial. Aviation runs on the fact that you can forecast fairly accurately into the next hour most of the time. The difficulty scales superlinearly over time due to the error accumulation over predictions

empath-nirvana 3y ago

> Neural networks are notoriously bad at exact approximation. Neural networks can compute pretty much anything. There's no reason, given the same inputs and with enough trainining data that it shouldn't be able to discover the same physical laws that were hard-coded previously.

T-A 3y ago

https://www.science.org/content/article/models-galaxies-atoms-simple-ai-shortcuts-speed-simulations-billions-times https://www.science.org/content/article/models-galaxies-atom... https://towardsdatascience.com/physics-informed-neural-networks-pinns-an-intuitive-guide-fff138069563 https://towardsdatascience.com/physics-informed-neural-netwo... https://maziarraissi.github.io/PINNs/ https://maziarraissi.github.io/PINNs/ https://arxiv.org/abs/2001.08055 https://arxiv.org/abs/2001.08055 https://arxiv.org/abs/2009.11990 https://arxiv.org/abs/2009.11990

amelius 3y ago

It's more like table lookup or interpolation than actual computation.

cowboysauce 3y ago

> So apparently the AI found some shortcut for doing the actual computational work. That is also surprising as weather is a chaotic system. Shortcuts should not exist. Why do you say that shortcuts should not exist? Even very basic statements like "falling pressure and increasing humidity indicate a storm is coming" are generally valid. I've done a little bit of storm-chasing and I'm able to point out areas that are likely to experience severe thunderstorms based on a few values (CAPE, dew point, wind shear, etc). I'm sure forecast meteorologists have even better skills. Are those not shortcuts?