6 ms·
No physics? No problem. AI weather forecasting is already making strides
- 1024core 2y agoHow come their weather balloons don't run afoul of local flight regulations?
- noobermin 2y agoKey detail: > The first step is potentially changing the way data is assimilated into AI-based models. At present, they almost universally use a set of initial conditions produced by a physics model. That is, a model like the ECMWF spends an enormous amount of computing power to collect data from buoys, surface stations, weather balloons, airplanes, ships, satellites, and many other sources and then synthesizes a set of initial conditions for grid points across the planet. All models then take this as the beginning "state" of the planet's weather and forecast from that. So this is essentially learning the time-stepping part of the physical model, not deriving predictions from raw data. While still interesting and probably still complex, this is far less impressive than the title lead me to believe.
- ToucanLoucan 2y agoI just assume every AI headline is one damn company or another trying to juice their stock price by finding someplace in their product line to shove an LLM. I'm right more often than not.
- tauroid 2y agoYou think the difficult part is merging observations with the last forecast? I guess it's a very underdetermined problem, but isn't the loss function (compare the forecast grid with later observations) the same whether you're doing grid_t0 -> grid_t1 or (observations, grid_t0) -> grid'_t0 -> grid_t1? I don't know enough about ML to know how much complexity the extra step adds, but doesn't seem like a massive difference.
- dannyz 2y agoObservation assimilation is a huge field in and of itself. Observables have biases that have to be included in assimilation, they also have finite resolution and so observation operators need to be taken into account.
- lainga 2y ago> For example, deep learning weather models have proven to be excellent at forecasting the tracks of hurricanes. But while these models are better at predicting where hurricanes will go, they tend to be lower-performing on the intensity changes of such storms relative to physics-based models. For context, intensity changes are the current Big Problem. Otis [1] had its track predicted almost exactly, but its explosive intensification from a tropical storm to a Cat 5 was totally unpredicted. Possibly some of the ~$12bn damage could have been avoided if Mexico had known that in advance. I've said this before in another comment some months back, but I'll repeat: my worry is that these models aren't learning some comprehensive new climate dynamics model with parameterisations [2] , but only fitting what the Earth has historically done. And if AI weather prediction is only learning what climate dynamics do 95% of the time, it's almost by definition not useful for predicting extreme weather and it will get less accurate the more the climate changes. You're just going to get more Otises. [1] https://en.wikipedia.org/wiki/Hurricane_Otis https://en.wikipedia.org/wiki/Hurricane_Otis [2] much as I would welcome, with open arms, some accurate AI-generated black-box parameterisations for e.g. subgrid precipitation - might be more explainable than the FORTRAN black-box parameterisations we have now :)*
- nomel 2y agoIs this a problem of observability? My naive understanding is that the majority of temperature data comes from where humans are: the surface. Hurricanes are 3d, extending up for miles. The models go almost entirely off the surface temperatures, with very very sparse balloon data (which is a poor sample, since a balloon will follow the air it's put in). Wouldn't the whole volume, or at least a little of it, need to be observed, since the energy in that volume is what's powering the hurricane, not the energy on the surface? I would assume this is why the models have trouble.
- cameldrv 2y agoSatellites also measure the temperature/height of clouds, and there's also some data from aircraft. A lot of commercial aircraft automatically report the temperature/pressure as they fly. The only problem is that a lot of their flight is in the stratosphere, but they give good data in their climb/descent.
- nico 2y agoAt some point we always need some sort of model of the world, you don’t want to call it physics?, that’s fine, but you are still modeling the world Interestingly though, physical models are usually expressed as mathematical equations. Which is an arbitrary way of modeling A Neural Network could technically “discover” different models, just through optimizing predictions for whatever we want the model to do We might not be able to distill the NN into nice compact equations, but they might still form a pretty good model of whatever phenomenon is fed to it through observational data Note that just by picking the input data and the output expectations, we are already defining a model
- universa1 2y agoin some sense yes, and if you are only interested in good predictions this might work out well. What, maybe due to my limited understanding, is, that this is not theory driven and therefore does not really provide understanding of the underlying process.
- bongodongobob 2y agoI think we understand the processes just fine. The issue is that we can't get a "closed form" solution for 10^100 interactions per planck time.
- gowld 2y agoThat's like saying chemistry has nothing to understand because we know Schrodinger's wave equation. Or that we understand biology and psychology just for for the same reason.
- bongodongobob 2y agoYeah, I think that makes sense. Those systems are similar in that they are made of zillions of tiny parts and there's no way to pull a one equation to rule them all out of it. We could, given infinite compute, but it's just not feasible.
- 2y ago
- lowbloodsugar 2y agoIt’s time for the wac wac wacky weather!
- bilsbie 2y agoAll physics is actually models of the world. So this is as much physics as newtons laws are.
- afiori 2y agoFrom a modeling point of view it likely has order of magnitude of order of magnitude of too many parameters to be a "good" physics model. The standard model is considered to have too many free variables and those are few enough that you can memorize and describe them all.
- mc_maurer 2y agoI've used this example when teaching students about models used for predictive power vs. models used to better understand mechanisms: You have a model that predicts with great accuracy that rowdy teens will TP your house this Friday night, so you sit up late waiting to scare them off. You have a model with less predictive power, but more discernible parameters. It tells you the parameter for whether or not houses have their front lights turned on has a high impact on likelihood of TP. You turn your front lights on and go to bed early. Sometimes we want models that produce highly accurate predictions, sometimes we want models that provide mechanistic insights that allow for other types of action. They're different simplifications/abstractions of reality that have their time and place, and can lead you astray in their own ways.
- danielmarkbruce 2y agoGreat example, great way to explain it. Nice work.
- mc_maurer 2y agoThank you!
- narrator 2y agoCan't you run the model in reverse? Brute force through various random parameters to the model to figure out which ones make a difference? Sure, it could have absurd dimensionality, but then it would be unlikely one could even grasp how to begin. After all, AlphaGo couldn't write a book for Humans about how to play go as well as it can.
- pona-a 2y agoThat's what model interpretability research is. You can train an interpretable model from the uninterpretable teacher, you can look at layer activations and how they correspond to certain features, or apply a hundred other domain-specific methods depending on your architecture. [0] Sadly, insight is always lost. In a noisy world where even with the best regularization, some fitting on it, or higher order features that describe it, is inevitable for maximizing prediction accuracy, especially if you don't have the right tools to model it (like transformers adapting to lacking registers [1]) and yet a lot of parameters within chosen architecture. What's worse, bad expectations are often much worse than none. If your loan had been denied by a fully opaque black box, you may be offered recourse to get an actual human on the case. If they've trained an interpretable student [2], either by intentional manipulation or by pure luck, it may have obscured the effect of some meta-feature likely corresponding to something like race, thus whitewashing the stochastically racist black box. [3] [0] "Interpretability in ML: A Broad Overview" https://www.lesswrong.com/posts/57fTWCpsAyjeAimTp/interpretability-in-ml-a-broad-overview-2 https://www.lesswrong.com/posts/57fTWCpsAyjeAimTp/interpreta... [1] "Thread: Circuits" https://distill.pub/2020/circuits/ https://distill.pub/2020/circuits/ [2] "Why Should I Trust You?": Explaining the Predictions of Any Classifier" https://arxiv.org/abs/1602.04938 https://arxiv.org/abs/1602.04938 [3] "Fairwashing: the risk of rationalization" https://proceedings.mlr.press/v97/aivodji19a https://proceedings.mlr.press/v97/aivodji19a
- robbomacrae 2y agoInteresting timing. Microsoft Aurora[0] was announced yesterday although that got very little attention here[1]. Too recent to be added to the article. [0]: https://www.microsoft.com/en-us/research/blog/introducing-aurora-the-first-large-scale-foundation-model-of-the-atmosphere/ https://www.microsoft.com/en-us/research/blog/introducing-au... [1]: https://news.ycombinator.com/item?id=40564138 https://news.ycombinator.com/item?id=40564138