5 ms·

I’m actually surprised to hear that the standard approaches used for weather forecasts do not use AI. One would think this would be one of the first areas that

by opdahl 3y ago

I’m actually surprised to hear that the standard approaches used for weather forecasts do not use AI. One would think this would be one of the first areas that would adopt AI given the data available. Also that these new AI systems are performing at the same level of the old non-AI approaches I feel is a real testament to the developers of those systems. This is not something we see in many other cases.

troupo 3y ago
> I’m actually surprised to hear that the standard approaches used for weather forecasts do not use AI. Why would they use AI where weather models exist, and work? What exactly would AI bring into space?
- deleted 3y ago
  [deleted]
- NoahECampbell 3y ago
  I mean as we're seeing right here, they are just as good in their testing stage while not taking several hours to run. They're more efficient to run and calculate since they don't need to go through a lengthy process to determine the weather, just take advantage of the cyclic nature of nature to do the work for us.
  - crote 3y ago
    The problem is that we are entering an era of rapid climate shift. Relying primarily on historical data doesn't work that well when you are trying to predict unprecedented weather events. The laws of physics, on the other hand, have not changed - so a simulation will still work just fine.
    - NoahECampbell 3y ago
      No we aren't actually. We are still well within historical data, since we use that still to calculate future weather patterns, and there hasn't been anything unprecedented that would trip up a model like this either.
      - zerodensity 3y ago
        So global warming is just a hoax? Or what are you saying?
        pixl97 3y ago
        No, what they are saying is events are not so far out of bounds that the models fall apart. Either there have been no black swans spotted yet, or in conjunction with other numerical models the outliers are correctly predicted.
        troupo 3y ago
        No. What they are saying is that: - weather is not climate - we are not in a situation where weather models don't work
- PaulHoule 3y ago
  A lot of people are using neural network models to model fields in variable resolution, solve PDEs and such. I know a bit about astrophysical modeling and you sure can't afford to model every part of a star in the same amount of detail. You can pick some special coordinate system or make the grid vary in size or you can use a neural network to model some f(x,y,z) and the network can allocate its own resources to put more detail where you need it. https://arxiv.org/abs/2102.11802 https://arxiv.org/abs/2102.11802
  - tnecniv 3y ago
    PINNs are neat but by my understanding they’re conceptually not too different from traditional numerical solution methods. The solution to your PDE is some complicated function that is not available in closed-form, so you require a numerical approximation of that function. Traditional methods use various kinds of grid discretization systems (possibly with interpolation schemes). PINNs replace the grid with a parametric continuous function. I’m not sure how PINNs handle discontinuities like shocks though.
Cthulhu_ 3y ago
They spent billions and decades on software and models that work good enough; I think AI would work too, given enough data to learn, but would it be better or just different? I mean keeping in mind the number of variables involved and, more recently, rapid climate change, el niño and forest fires that are much more difficult for an AI to keep in mind (I think). It's a competitor to existing models and supercomputers, but it has to prove its reliability first.
supdudesupdude 3y ago
I'm actually not surprised that people think 'AI' can fix everything including weather prediction. I've heard about AI machine learning forecasts for 10+ years now, waiting for one of them to be actually used in industry... This aint a retail product recommendation. If the AI makes a bad choice it will cost millions/billions of dollars and we could lose many MANY lives.
- counters 3y ago
  > I've heard about AI machine learning forecasts for 10+ years now, waiting for one of them to be actually used in industry... The problem is that they offer little to no advantage over the highly optimized ML-based forecast post-processing systems widely in use in the industry. You see an awful lot of hype from start-ups proclaiming their AI forecasts are "the most accurate ever"... when in reality they barely improve at all over the status quo that can be achieved with rather simple statistical modeling.
counters 3y ago
ML/AI has been used in the weather forecasting world since the 1970's - most often to post-process forecast model output to better calibrate it against observations and correct for biases. > One would think this would be one of the first areas that would adopt AI given the data available. Actually, the challenge has been that there _isn't_ enough data available. Sure, we have lots of satellite observations and many other sources, but none of these paint a holistic picture of the atmosphere of the sort you'd need to actually forecast the weather with any precision. ERA-5 - a model-based "re-analysis" that assimilates many observations and tries to create a coherent picture - is only a few years old, and has been the keystone that unlocked all of this development over the past few years.

by opdahl 3y ago

troupo 3y ago

> I’m actually surprised to hear that the standard approaches used for weather forecasts do not use AI. Why would they use AI where weather models exist, and work? What exactly would AI bring into space?

deleted 3y ago
[deleted]
NoahECampbell 3y ago
I mean as we're seeing right here, they are just as good in their testing stage while not taking several hours to run. They're more efficient to run and calculate since they don't need to go through a lengthy process to determine the weather, just take advantage of the cyclic nature of nature to do the work for us.
- crote 3y ago
  The problem is that we are entering an era of rapid climate shift. Relying primarily on historical data doesn't work that well when you are trying to predict unprecedented weather events. The laws of physics, on the other hand, have not changed - so a simulation will still work just fine.
  - NoahECampbell 3y ago
    No we aren't actually. We are still well within historical data, since we use that still to calculate future weather patterns, and there hasn't been anything unprecedented that would trip up a model like this either.
    - zerodensity 3y ago
      So global warming is just a hoax? Or what are you saying?
      - pixl97 3y ago
        No, what they are saying is events are not so far out of bounds that the models fall apart. Either there have been no black swans spotted yet, or in conjunction with other numerical models the outliers are correctly predicted.
      - troupo 3y ago
        No. What they are saying is that: - weather is not climate - we are not in a situation where weather models don't work
PaulHoule 3y ago
A lot of people are using neural network models to model fields in variable resolution, solve PDEs and such. I know a bit about astrophysical modeling and you sure can't afford to model every part of a star in the same amount of detail. You can pick some special coordinate system or make the grid vary in size or you can use a neural network to model some f(x,y,z) and the network can allocate its own resources to put more detail where you need it. https://arxiv.org/abs/2102.11802 https://arxiv.org/abs/2102.11802
- tnecniv 3y ago
  PINNs are neat but by my understanding they’re conceptually not too different from traditional numerical solution methods. The solution to your PDE is some complicated function that is not available in closed-form, so you require a numerical approximation of that function. Traditional methods use various kinds of grid discretization systems (possibly with interpolation schemes). PINNs replace the grid with a parametric continuous function. I’m not sure how PINNs handle discontinuities like shocks though.

Cthulhu_ 3y ago

They spent billions and decades on software and models that work good enough; I think AI would work too, given enough data to learn, but would it be better or just different? I mean keeping in mind the number of variables involved and, more recently, rapid climate change, el niño and forest fires that are much more difficult for an AI to keep in mind (I think). It's a competitor to existing models and supercomputers, but it has to prove its reliability first.

supdudesupdude 3y ago

I'm actually not surprised that people think 'AI' can fix everything including weather prediction. I've heard about AI machine learning forecasts for 10+ years now, waiting for one of them to be actually used in industry... This aint a retail product recommendation. If the AI makes a bad choice it will cost millions/billions of dollars and we could lose many MANY lives.

counters 3y ago
> I've heard about AI machine learning forecasts for 10+ years now, waiting for one of them to be actually used in industry... The problem is that they offer little to no advantage over the highly optimized ML-based forecast post-processing systems widely in use in the industry. You see an awful lot of hype from start-ups proclaiming their AI forecasts are "the most accurate ever"... when in reality they barely improve at all over the status quo that can be achieved with rather simple statistical modeling.

counters 3y ago

ML/AI has been used in the weather forecasting world since the 1970's - most often to post-process forecast model output to better calibrate it against observations and correct for biases. > One would think this would be one of the first areas that would adopt AI given the data available. Actually, the challenge has been that there _isn't_ enough data available. Sure, we have lots of satellite observations and many other sources, but none of these paint a holistic picture of the atmosphere of the sort you'd need to actually forecast the weather with any precision. ERA-5 - a model-based "re-analysis" that assimilates many observations and tries to create a coherent picture - is only a few years old, and has been the keystone that unlocked all of this development over the past few years.