3 ms·

When I mentioned here some time ago, that maybe we shouldn't be trusting our climate science models so much, as they consist of millions of lines of poorly test

When I mentioned here some time ago, that maybe we shouldn't be trusting our climate science models so much, as they consist of millions of lines of poorly tested code, I was heavily criticized...

moultano 7y ago
Not trusting them is reasonable, but then where does that leave you? You don't know then whether they're underestimating or overestimating things. I'm strongly in favor of building a prior from the geological record and leaning on that as much or more than the models, but that doesn't lead to a very different conclusion.
- burntoutfire 7y ago
  I guess that, at least compared to society in general, I'm a cognitive nihilist? I highly doubt that it is in our power to predict with useful accuracy something as complex as climate. If I'm not mistaken, we're still having problems modelling stuff a billion times less complex than climate (such as an air flow in turbines etc.)...
  - moultano 7y ago
    Generally, more uncertainty around climate change means you should be more supportive of climate change mitigation rather than less, because it means we can't rule out things that look like the P-T extinction.
    - burntoutfire 7y ago
      There's infinite number of events which we can't rule out. My approach is basically the opposite - if the evidence for something is super-sketchy, maybe we shouldn't spend trillions (or even quadrillions?) on preparing for it.
      - moultano 7y ago
        The evidence is completely airtight. The problem is with the precision of predictions, not the evidence for the mechanism and rough magnitude.
btrettel 7y ago
Computational fluid dynamics (CFD, of which climate modeling is a subfield) actually does software testing better than other scientific fields in my experience. That's not to say that all CFD models are well tested, just that they're doing a fair bit more than nothing. Typically testing is divided into "verification", which is checking the math, and "validation" which is comparing against experimental data. If you can't find anything about a model's verification and validation, don't trust it. Also, it's actually fairly easy to get apparently good validation by cherry picking the right data, so even doing well here isn't enough. You basically need to be an expert to know the right test cases to look at... Here's a slide deck that shows the (not great) state of typical practice in validation: https://www.osti.gov/biblio/1141709 https://www.osti.gov/biblio/1141709 Note that some of the "better" journals do worse... Here's a public domain CFD software that I think does a good job on verification and validation: https://github.com/firemodels/fds https://github.com/firemodels/fds Full disclosure: I have contributed to this project.
crocowhile 7y ago
You were heavily criticized because you said nonsense. Cancer research also relies on millions of lines of code but you would not say the same, would you? The science of climate change goes well beyond the models. The models account for only a small fraction of the science and they are produced completely independently by many research groups, leading to overlapping results.

When I mentioned here some time ago, that maybe we shouldn't be trusting our climate science models so much, as they consist of millions of lines of poorly tested code, I was heavily criticized...

moultano 7y ago

Not trusting them is reasonable, but then where does that leave you? You don't know then whether they're underestimating or overestimating things. I'm strongly in favor of building a prior from the geological record and leaning on that as much or more than the models, but that doesn't lead to a very different conclusion.

burntoutfire 7y ago
I guess that, at least compared to society in general, I'm a cognitive nihilist? I highly doubt that it is in our power to predict with useful accuracy something as complex as climate. If I'm not mistaken, we're still having problems modelling stuff a billion times less complex than climate (such as an air flow in turbines etc.)...
- moultano 7y ago
  Generally, more uncertainty around climate change means you should be more supportive of climate change mitigation rather than less, because it means we can't rule out things that look like the P-T extinction.
  - burntoutfire 7y ago
    There's infinite number of events which we can't rule out. My approach is basically the opposite - if the evidence for something is super-sketchy, maybe we shouldn't spend trillions (or even quadrillions?) on preparing for it.
    - moultano 7y ago
      The evidence is completely airtight. The problem is with the precision of predictions, not the evidence for the mechanism and rough magnitude.

btrettel 7y ago

Computational fluid dynamics (CFD, of which climate modeling is a subfield) actually does software testing better than other scientific fields in my experience. That's not to say that all CFD models are well tested, just that they're doing a fair bit more than nothing. Typically testing is divided into "verification", which is checking the math, and "validation" which is comparing against experimental data. If you can't find anything about a model's verification and validation, don't trust it. Also, it's actually fairly easy to get apparently good validation by cherry picking the right data, so even doing well here isn't enough. You basically need to be an expert to know the right test cases to look at... Here's a slide deck that shows the (not great) state of typical practice in validation: https://www.osti.gov/biblio/1141709 https://www.osti.gov/biblio/1141709 Note that some of the "better" journals do worse... Here's a public domain CFD software that I think does a good job on verification and validation: https://github.com/firemodels/fds https://github.com/firemodels/fds Full disclosure: I have contributed to this project.

crocowhile 7y ago

You were heavily criticized because you said nonsense. Cancer research also relies on millions of lines of code but you would not say the same, would you? The science of climate change goes well beyond the models. The models account for only a small fraction of the science and they are produced completely independently by many research groups, leading to overlapping results.