4 ms·

Simulating the pandemic: What Covid forecasters can learn from climate models

22 points by hatmatrix 6y ago

cryptica 6y ago
> because researchers didn’t appreciate how sensitive CovidSim was to small changes in its inputs, their results overestimated As a software engineer who has written simulations before, I can assert that this is almost always the case with simulations. The more complex the phenomena is that you're trying to model, the more likely it is that the simulation is completely wrong because you got a single parameter slightly off. That's why I'm skeptical of climate simulations too because they are insanely complex and they're certainly missing a lot of parameters. For example, just off the top of my head, I wonder if climate models factor in the fact that tree populations will evolve to adapt to increased CO2... I don't think they can model evolutions of tree populations because it gets into genetics. Which is a level of complexity far beyond what even the top supercomputers can simulate and even then, a lot of things would have to be simplified to the point that the model ends up having nothing to do with the real world. The butterfly effect is one of the reasons why complex simulations are not useful in predicting anything. Simulations of complex phenomena are completely useless IMO. You can always cherry pick one simulation out of thousands which is reliable for a few years in a row but they fall apart sooner and later so you end up using a different simulation every few years... And people keep coming up with excuses every time saying that the supercomputers were not fast enough the year before.
- nn3 6y ago
  For climate the simulations can at least back tested against the past climate. It's unclear how that can be done for for Covid. There's nothing to test against because both the virus and the society's reaction is very different.
  - cryptica 6y ago
    But the back testing is useless because you can just have a lot of models and cherry pick all those models which happen to match historical data. Then each year some of those models are proven wrong and the attention shifts towards models which happened to get it right... And new models are invented constantly. If you have enough models and companies working on them, you will always find a few which just happen to have guessed all the numbers correctly. It's like lottery. Out of millions of people, some people will get all the numbers right.
    - nn3 6y ago
      Yes, back testing is not perfect, but it's better than nothing. In the Covid simulation case they have nothing as far as I can tell.
- the8472 6y ago
  > the fact that tree populations will evolve to adapt to increased CO2 Many climate models only aim to predict the next century or two because that's what's relevant for humans. Trees have long generation cycles and evolution tends to take many generations to show significant effects. So on geological timescales they will probably adapt, but considering the speed at which AGW is happening that adaption will be too late to be of interest.
bioinformatics 6y ago
> What COVID forecasters can learn from climate models That they don't work? I have been working with simulation and models for the past 25 years, and models are just that models. They provide a picture of what you want to see, not the truth.
- bb1234 6y ago
  Do you have an alternative to suggest?
  - giardini 6y ago
    He's not saying that models should not be used but that they should be used thoughtfully. Brainstorming in groups helps at first, but later sessions where you play Devil's Advocate to critique models and/or re-evaluate are also useful.
zyxzevn 6y ago
Climate models? They are so extremely bad, scientifically.
coldcode 6y ago
The headline ignores the fact that the covid forecasters have changed their model over time to be basically what the paper said to (based on the initial model from March done in great panic). So this story is really nothing. Models improve over time if you have enough time.
Pensacola 6y ago
I'll guess not much - since the climate models have been consistently wrong. But since Covid was politically weaponized in 2020 just as climate science was politically weaponized in the 1990s, I suppose we can learn something about Covid forecasters from their climate counterparts.
- ericb 6y ago
  > politically weaponized If a political leader's response to a quarter million deaths is denial, that's not weaponization, that's democracy working.
  - deleted 6y ago
    [deleted]

22 points by hatmatrix 6y ago

> because researchers didn’t appreciate how sensitive CovidSim was to small changes in its inputs, their results overestimated As a software engineer who has written simulations before, I can assert that this is almost always the case with simulations. The more complex the phenomena is that you're trying to model, the more likely it is that the simulation is completely wrong because you got a single parameter slightly off. That's why I'm skeptical of climate simulations too because they are insanely complex and they're certainly missing a lot of parameters. For example, just off the top of my head, I wonder if climate models factor in the fact that tree populations will evolve to adapt to increased CO2... I don't think they can model evolutions of tree populations because it gets into genetics. Which is a level of complexity far beyond what even the top supercomputers can simulate and even then, a lot of things would have to be simplified to the point that the model ends up having nothing to do with the real world. The butterfly effect is one of the reasons why complex simulations are not useful in predicting anything. Simulations of complex phenomena are completely useless IMO. You can always cherry pick one simulation out of thousands which is reliable for a few years in a row but they fall apart sooner and later so you end up using a different simulation every few years... And people keep coming up with excuses every time saying that the supercomputers were not fast enough the year before.

nn3 6y ago
For climate the simulations can at least back tested against the past climate. It's unclear how that can be done for for Covid. There's nothing to test against because both the virus and the society's reaction is very different.
- cryptica 6y ago
  But the back testing is useless because you can just have a lot of models and cherry pick all those models which happen to match historical data. Then each year some of those models are proven wrong and the attention shifts towards models which happened to get it right... And new models are invented constantly. If you have enough models and companies working on them, you will always find a few which just happen to have guessed all the numbers correctly. It's like lottery. Out of millions of people, some people will get all the numbers right.
  - nn3 6y ago
    Yes, back testing is not perfect, but it's better than nothing. In the Covid simulation case they have nothing as far as I can tell.
the8472 6y ago
> the fact that tree populations will evolve to adapt to increased CO2 Many climate models only aim to predict the next century or two because that's what's relevant for humans. Trees have long generation cycles and evolution tends to take many generations to show significant effects. So on geological timescales they will probably adapt, but considering the speed at which AGW is happening that adaption will be too late to be of interest.

bioinformatics 6y ago

> What COVID forecasters can learn from climate models That they don't work? I have been working with simulation and models for the past 25 years, and models are just that models. They provide a picture of what you want to see, not the truth.

bb1234 6y ago
Do you have an alternative to suggest?
- giardini 6y ago
  He's not saying that models should not be used but that they should be used thoughtfully. Brainstorming in groups helps at first, but later sessions where you play Devil's Advocate to critique models and/or re-evaluate are also useful.

zyxzevn 6y ago

Climate models? They are so extremely bad, scientifically.

coldcode 6y ago

The headline ignores the fact that the covid forecasters have changed their model over time to be basically what the paper said to (based on the initial model from March done in great panic). So this story is really nothing. Models improve over time if you have enough time.

Pensacola 6y ago

I'll guess not much - since the climate models have been consistently wrong. But since Covid was politically weaponized in 2020 just as climate science was politically weaponized in the 1990s, I suppose we can learn something about Covid forecasters from their climate counterparts.

ericb 6y ago
> politically weaponized If a political leader's response to a quarter million deaths is denial, that's not weaponization, that's democracy working.
- deleted 6y ago
  [deleted]