3 ms·

Yup. There's fundamentally no way to automatically verify predictive accuracy, the assumptions in ML papers about such accuracy is (the almost always false pres

Yup. There's fundamentally no way to automatically verify predictive accuracy, the assumptions in ML papers about such accuracy is (the almost always false presumption) that there are no distributional shifts in the data generating process. Here, since weather certainly changes its fundamental patterns over time, there is no way of reliably predicting out-sample performance. It's highly likely that in 1-2 years time we'd take all physics-based predictions and all predictions of current AI models, and find the AI model accuracy drops off a cliff when faced with a real out-sample.

wcoenen 2y ago
Weather models can't compute some of the important physics that happens at sub-grid scale, and use various tricks to deal with that. How sure are you that these traditional models aren't also heavily tuned on past data to maximize performance? Perhaps they will also perform badly "out of distribution".
- mjburgess 2y ago
  Causal physical models have no distributional requirements, so they are not sensitive to distribution shifts. ie., A causal model accounts for all possible distributions. The kind of risk with a causal model is that the model itself, of reality, is incorrect. This is a different kind of risk than there being 'no model at all' as in the case of curve-fitting over historical data, which is radically more fragile to expected shifts. In general, we're quite good at knowing the limitations of causal models, ie., specifying model risk here is much easier. You even, exactly, point out a known problem in the modelling which is impossible to state for an AI model. Since the AI model is just a weak induction, there are no terms/structures/etc. within that model which can be analysed to understand what parts of it are sensitive to what aspects of the system from which the data was taken. All we can say is, we know that in general, train/test distributions have to be "nearly exactly the same" for any of these methods to show anything like cross-val levels of accuracy. So we can very confidently predict when we know train/test wont be the same, that this is a mumbojumbo metric. Indeed, in the vast majority of common ML examples you can, right now, just go and look at real out-sample data collected later than the "common dataset" and you'll find the val accuracy is random or worse-than-random despite arbitarily high cross-val scores. The dataset which drives me most up-the-wall on this is house price datasets, or pricing datasets in general. Prices, generally, follow geometric brownian motion and nearly all ML models are extremelhy bad at modelling prices. So it's basically pseudoscience whenver anyone uses these datasets to demonstrate anything, esp. predictive accuracy.
staunton 2y ago
Ultimately, a hybrid approach might win out in the end. Use equations for what we know, use machine learning to fit parameters in the equations, as well as terms we don't know. Prople nowadays can solve differential equations where some terms are "neural networks" and train those networks on data while numerically solving the equation. (some people call it "neural differential equations", if you want a search query to start)
counters 2y ago
> Here, since weather certainly changes its fundamental patterns over time, there is no way of reliably predicting out-sample performance. This seems overly pessimistic. The weather doesn't change its fundamental patterns over time; those are governed by the fluid dynamics of the atmosphere, and are largely forced (to first order) by things like the amount of solar radiation received by the sun and the the configuration of the continents - neither of which change much at all on timescales of a few thousand or tens of thousands of years. The total manifold of "realistic" weather patterns - especially trajectories of weather patterns conditioned on one or more prior weather sates - is vastly smaller than it would seem at first glance (there's only so much variation in how a Rossby wave can evolve and there is clear asymptotic behavior with regards to amplitude). I think if you wanted an "ultimate stress test" of an AI weather forecasting system, you could run a a high-fidelity general circulation model equilibrated to the current climate freely for a several thousand year cycle, and randomly draw initial conditions to branch off a 10-day forecast. It shouldn't be _perfect_ (there is likely meaningful skew between the atmospheric state trajectories that arise in this free running simulation compared to reanalysis cycled over the historical observation record for a few decades), but the trends in error growth should be meaningful and directly interpretable in the context of a real-world forecast.
- mjburgess 2y ago
  The causal mechanics are the same. ML works on very specific distributions of measurements of those patterns. ML models are models of measures, not of the causal process which generates those measures. In this sense they arent 'models' at all, in the traditional scientific sense of the term.

wcoenen 2y ago

Weather models can't compute some of the important physics that happens at sub-grid scale, and use various tricks to deal with that. How sure are you that these traditional models aren't also heavily tuned on past data to maximize performance? Perhaps they will also perform badly "out of distribution".

mjburgess 2y ago
Causal physical models have no distributional requirements, so they are not sensitive to distribution shifts. ie., A causal model accounts for all possible distributions. The kind of risk with a causal model is that the model itself, of reality, is incorrect. This is a different kind of risk than there being 'no model at all' as in the case of curve-fitting over historical data, which is radically more fragile to expected shifts. In general, we're quite good at knowing the limitations of causal models, ie., specifying model risk here is much easier. You even, exactly, point out a known problem in the modelling which is impossible to state for an AI model. Since the AI model is just a weak induction, there are no terms/structures/etc. within that model which can be analysed to understand what parts of it are sensitive to what aspects of the system from which the data was taken. All we can say is, we know that in general, train/test distributions have to be "nearly exactly the same" for any of these methods to show anything like cross-val levels of accuracy. So we can very confidently predict when we know train/test wont be the same, that this is a mumbojumbo metric. Indeed, in the vast majority of common ML examples you can, right now, just go and look at real out-sample data collected later than the "common dataset" and you'll find the val accuracy is random or worse-than-random despite arbitarily high cross-val scores. The dataset which drives me most up-the-wall on this is house price datasets, or pricing datasets in general. Prices, generally, follow geometric brownian motion and nearly all ML models are extremelhy bad at modelling prices. So it's basically pseudoscience whenver anyone uses these datasets to demonstrate anything, esp. predictive accuracy.

staunton 2y ago

Ultimately, a hybrid approach might win out in the end. Use equations for what we know, use machine learning to fit parameters in the equations, as well as terms we don't know. Prople nowadays can solve differential equations where some terms are "neural networks" and train those networks on data while numerically solving the equation. (some people call it "neural differential equations", if you want a search query to start)

counters 2y ago

> Here, since weather certainly changes its fundamental patterns over time, there is no way of reliably predicting out-sample performance. This seems overly pessimistic. The weather doesn't change its fundamental patterns over time; those are governed by the fluid dynamics of the atmosphere, and are largely forced (to first order) by things like the amount of solar radiation received by the sun and the the configuration of the continents - neither of which change much at all on timescales of a few thousand or tens of thousands of years. The total manifold of "realistic" weather patterns - especially trajectories of weather patterns conditioned on one or more prior weather sates - is vastly smaller than it would seem at first glance (there's only so much variation in how a Rossby wave can evolve and there is clear asymptotic behavior with regards to amplitude). I think if you wanted an "ultimate stress test" of an AI weather forecasting system, you could run a a high-fidelity general circulation model equilibrated to the current climate freely for a several thousand year cycle, and randomly draw initial conditions to branch off a 10-day forecast. It shouldn't be _perfect_ (there is likely meaningful skew between the atmospheric state trajectories that arise in this free running simulation compared to reanalysis cycled over the historical observation record for a few decades), but the trends in error growth should be meaningful and directly interpretable in the context of a real-world forecast.

mjburgess 2y ago
The causal mechanics are the same. ML works on very specific distributions of measurements of those patterns. ML models are models of measures, not of the causal process which generates those measures. In this sense they arent 'models' at all, in the traditional scientific sense of the term.