6 ms·

Yeah, this could be survivorship bias: https://motls.blogspot.com/2017/03/selection-of-climate-model-survivors.html https://motls.blogspot.com/2017/03/selection

Yeah, this could be survivorship bias: https://motls.blogspot.com/2017/03/selection-of-climate-model-survivors.html https://motls.blogspot.com/2017/03/selection-of-climate-mode... As someone who's written basic fluid dynamics codes, I'm blown away by the complexity of climate models. The errors I ran into in debugging my relatively straightforward codes were incredibly subtle. Like pressures, velocities still "looking right" but still being wrong, due to numerical viscosity and other artifacts of the inexact solutions to Navier-Stokes. This is for flow in a box in a steady state. For an entire planetary atmosphere at a time scale of decades, I can't even imagine how they can track down every error without duct taping "corrections" (overfitting) on everything.

IanCal 7y ago
Doesn't this partly fall into the weather Vs climate difference? I can't run fluid dynamics in my head but no matter how much you shake up the box once it's on the table I can say "I think the water/oil/honey will end up at the bottom". But also yes, it's complex, that's why independent groups dedicate huge amounts of time researching and building them and then models are compared against each other.
- telotortium 7y ago
  > Doesn't this partly fall into the weather Vs climate difference? I can't run fluid dynamics in my head but no matter how much you shake up the box once it's on the table I can say "I think the water/oil/honey will end up at the bottom". Not in the slightest. For example, the output of a numerically unstable [1] algorithm, run on 64-bit floats, can diverge arbitrarily from the output of the same algorithm run on real numbers with infinite precision (which computers can't do). Often this will lead to obviously wrong outputs, but not always. There are numerically-stable versions of most of the workhorse linear algebra algorithms, but (1) sometimes it's possible to use a faster, numerically unstable algorithm, and this is a matter of judgment that can be mistaken and (2) it's easy to introduce numerical instability via subtle bugs, or even via the exact arithmetic steps you use to calculate a result (in particular, trying to add or subtract two numbers of very different orders of magnitude is a big no-no). [1] https://en.wikipedia.org/wiki/Numerical_stability https://en.wikipedia.org/wiki/Numerical_stability
  - irq11 7y ago
    This is not a rebuttal to the parent. They’re talking about stability of prediction of aggregate properties, and you’re fixating on numerical precision errors and algorithm bugs in particular implementations. The one has little to do with the other. Numerical instability leads to obvious, wildly inconsistent errors. It isn’t subtle. Literally no climate simulations are going off the rails because of these kinds of first-year grad-student code problems.
throwyweather 7y ago
Correctness of numerical codes is a huge and under-appreciated concern in climate models. There have been some attempts to apply software correctness techniques to climate models. For example, https://people.engr.ncsu.edu/jwb/papers/altuntas-correctness-2018.pdf https://people.engr.ncsu.edu/jwb/papers/altuntas-correctness... Much more work in this direction is needed.
- skosch 7y ago
  ... but if you think, "hey, so the models could be wrong, how reassuring", think again. It's just as likely that reality is worse than predicted.
  - ekianjo 7y ago
    How can you say that unless we know how to sort good models from bad ones before observations kick in? As far as I know its always very easy to fit historical data with thousands of models but very few will actually be useful for prediction and we have no idea which ones.
  - thaumasiotes 7y ago
    > It's just as likely that reality is worse than predicted. This cannot possibly be correct; it's the same reasoning as the argument "Either the sun will rise tomorrow or it won't. These two possibilities exhaust the entire space. Therefore, the odds of the sun rising tomorrow are 50%, one in two." The premises are completely correct -- and unrelated to the conclusion.
    - moultano 7y ago
      It isn't the same reasoning at all. Our prior for what could possibly happen should be informed by what has happened before, and we know to the best of what geology can tell us, that many mass extinctions on earth were caused by CO2 mediated warming, including the one that wiped out 95% of all species at the end of the Permian. https://www.nytimes.com/2018/12/07/science/climate-change-mass-extinction.html https://www.nytimes.com/2018/12/07/science/climate-change-ma...
mturmon 7y ago
This blog post is highly misleading and poorly-informed. The author of it does not seem to know that climate scientists do test their models against reality, leading him/her to waste a lot of words explaining how cargo cult science is bad, and how overfitting is being done, etc. These points are valid in the abstract, but don't represent how real climate science works. Amazingly, the author at one point says: "In proper science, the different theories are bloodily competing with each other." as a contrast to climate science, where supposedly these models are just peacefully cooperating to raise more funds. Anyone who has been in the middle of a conference where people with competing models are comparing results (to each other and to observations) knows how off-base this is.
gameswithgo 7y ago
total heat content of the planet is way way simpler than fluid dynamics
- SlowRobotAhead 7y ago
  ... this is sarcasm? I honestly can’t tell. Can you elaborate your point of view a little more?
henrifdrake 7y ago
How can you distinguish survivorship bias different from... developing better models? If there are any models that made quantitative predictions of global temperature change (whether warming, cooling, or neither), please let me know. I am extremely interested :) P.S. Read the paper here: https://pubs.giss.nasa.gov/abs/ha08910q.html https://pubs.giss.nasa.gov/abs/ha08910q.html

IanCal 7y ago

Doesn't this partly fall into the weather Vs climate difference? I can't run fluid dynamics in my head but no matter how much you shake up the box once it's on the table I can say "I think the water/oil/honey will end up at the bottom". But also yes, it's complex, that's why independent groups dedicate huge amounts of time researching and building them and then models are compared against each other.

telotortium 7y ago
> Doesn't this partly fall into the weather Vs climate difference? I can't run fluid dynamics in my head but no matter how much you shake up the box once it's on the table I can say "I think the water/oil/honey will end up at the bottom". Not in the slightest. For example, the output of a numerically unstable [1] algorithm, run on 64-bit floats, can diverge arbitrarily from the output of the same algorithm run on real numbers with infinite precision (which computers can't do). Often this will lead to obviously wrong outputs, but not always. There are numerically-stable versions of most of the workhorse linear algebra algorithms, but (1) sometimes it's possible to use a faster, numerically unstable algorithm, and this is a matter of judgment that can be mistaken and (2) it's easy to introduce numerical instability via subtle bugs, or even via the exact arithmetic steps you use to calculate a result (in particular, trying to add or subtract two numbers of very different orders of magnitude is a big no-no). [1] https://en.wikipedia.org/wiki/Numerical_stability https://en.wikipedia.org/wiki/Numerical_stability
- irq11 7y ago
  This is not a rebuttal to the parent. They’re talking about stability of prediction of aggregate properties, and you’re fixating on numerical precision errors and algorithm bugs in particular implementations. The one has little to do with the other. Numerical instability leads to obvious, wildly inconsistent errors. It isn’t subtle. Literally no climate simulations are going off the rails because of these kinds of first-year grad-student code problems.

throwyweather 7y ago

Correctness of numerical codes is a huge and under-appreciated concern in climate models. There have been some attempts to apply software correctness techniques to climate models. For example, https://people.engr.ncsu.edu/jwb/papers/altuntas-correctness-2018.pdf https://people.engr.ncsu.edu/jwb/papers/altuntas-correctness... Much more work in this direction is needed.

skosch 7y ago
... but if you think, "hey, so the models could be wrong, how reassuring", think again. It's just as likely that reality is worse than predicted.
- ekianjo 7y ago
  How can you say that unless we know how to sort good models from bad ones before observations kick in? As far as I know its always very easy to fit historical data with thousands of models but very few will actually be useful for prediction and we have no idea which ones.
- thaumasiotes 7y ago
  > It's just as likely that reality is worse than predicted. This cannot possibly be correct; it's the same reasoning as the argument "Either the sun will rise tomorrow or it won't. These two possibilities exhaust the entire space. Therefore, the odds of the sun rising tomorrow are 50%, one in two." The premises are completely correct -- and unrelated to the conclusion.
  - moultano 7y ago
    It isn't the same reasoning at all. Our prior for what could possibly happen should be informed by what has happened before, and we know to the best of what geology can tell us, that many mass extinctions on earth were caused by CO2 mediated warming, including the one that wiped out 95% of all species at the end of the Permian. https://www.nytimes.com/2018/12/07/science/climate-change-mass-extinction.html https://www.nytimes.com/2018/12/07/science/climate-change-ma...

mturmon 7y ago

This blog post is highly misleading and poorly-informed. The author of it does not seem to know that climate scientists do test their models against reality, leading him/her to waste a lot of words explaining how cargo cult science is bad, and how overfitting is being done, etc. These points are valid in the abstract, but don't represent how real climate science works. Amazingly, the author at one point says: "In proper science, the different theories are bloodily competing with each other." as a contrast to climate science, where supposedly these models are just peacefully cooperating to raise more funds. Anyone who has been in the middle of a conference where people with competing models are comparing results (to each other and to observations) knows how off-base this is.

gameswithgo 7y ago

total heat content of the planet is way way simpler than fluid dynamics

SlowRobotAhead 7y ago
... this is sarcasm? I honestly can’t tell. Can you elaborate your point of view a little more?

henrifdrake 7y ago

How can you distinguish survivorship bias different from... developing better models? If there are any models that made quantitative predictions of global temperature change (whether warming, cooling, or neither), please let me know. I am extremely interested :) P.S. Read the paper here: https://pubs.giss.nasa.gov/abs/ha08910q.html https://pubs.giss.nasa.gov/abs/ha08910q.html