4 ms·

It reminds me of the covid models, particularly the infamous Imperial model. https://en.wikipedia.org/wiki/CovidSim https://en.wikipedia.org/wiki/CovidSim The

It reminds me of the covid models, particularly the infamous Imperial model. https://en.wikipedia.org/wiki/CovidSim https://en.wikipedia.org/wiki/CovidSim The question is whether it's useful to have a model at all, or whether it's better to just reason about it from first principles. If a basic simulation gave you a tool to explore emergent properties, perhaps some theories could be generalised to apply to real economies? Or maybe not, I don't know.

whatshisface 6y ago
Models are supposed to be based on first-principles reasoning. A "model" that's not designed to capture some aspect of reality is called a computer game...
SiempreViernes 6y ago
The Imperial model gives reproducible results, so most of the noise was professional programmers being salty about the fact that ugly code can still be accurate code. The fact that it predicted things like 80% of the population becoming infected was simply unrelated to code quality.
- benmller313 6y ago
  Regardless of code quality, are these models really useful if, as you say, they are "accurate", but still make predictions wildly out of step with reality?
  - whatshisface 6y ago
    The real imperial story is that there were two wrong parties: - The scientists who were not including enough detail in their math. - The drive-by developers who had no idea what did or didn't make scientific simulations work. Together they produced both an inaccurate estimate, and a github page full of uninformed speculation about what the problem might have been. ;)
    - SiempreViernes 6y ago
      Honestly I think the main problem was they choose overly pessimistic estimates, or maybe that their model is so expensive they could only investigate a few choices of the parameters.
  - Quarrelsome 6y ago
    same way that a given implementation is better than none. Its a concrete implementation that serves as an anchor. Its a fixed point to discuss implementation issues around as well as working as a harness that others can build on/around.
  - SiempreViernes 6y ago
    Usually, because they help narrow down the sources of error. If you've tested your code so you're sure the various bits work as intended, any remaining somewhat crazy results are down to things you don't understand properly. A somewhat validated code allows you to do by computational experiments what you can't by real life experiments. Very handy if one of the common end states in the experiments is "death".
- plantain 6y ago
  Wasn't it more professional programmers being salty that policy decisions were being made by code with absolutely no tests or peer review?
  - SiempreViernes 6y ago
    Code with no _unit tests_ or _code reviews_ possibly, it was obviously tested by the scientists that it made somewhat sensible and self-consistent predictions. In addition, predictions by earlier versions of the code had been peer reviewed, which is about as good as it gets outside of computer science. The report itself wasn't formally peer reviewed either, though in the crisis mentality of March this year not many people had the patience for peer review. Frankly, most still don't if you consider the amount of news about unreviewed science that get published.
- bart_spoon 6y ago
  Reproducibility doesn't mean accurate or good, it means it can be recreated. Its a crucial first step, but a model can be reproducible and also totally worthless.
  - SiempreViernes 6y ago
    Yeah, but as you can see in this thread the rock star programmers of HN only know enough epidemiology to complain about code formatting and improper RNG seeding.
- COVardsStayHome 6y ago
  The same model that predicted that Sweden will have 100 000 dead by the end of June? https://www.medrxiv.org/content/10.1101/2020.04.11.20062133v1 https://www.medrxiv.org/content/10.1101/2020.04.11.20062133v... The authors of this junk model should be Hanged, drawn and quartered.
- Jabbles 6y ago
  gives? perhaps it does now, but it was certainly non-determinstic when it was first released, which is also when it was used for policy decisions. See for example https://news.ycombinator.com/item?id=23099386 https://news.ycombinator.com/item?id=23099386 But how useful was the model? It showed that the virus would spread exponentially, but we knew that. Is there anything that it showed that we didn't know when they programmed it, with constants tuned to what they expected?
  - cinntaile 6y ago
    That was the point. They used stochastic partial differential equations, they're by definition non-deterministic.
    - Jabbles 6y ago
      No, they should be reproducible if run with the same seed.
mmcwilliams 6y ago
An interesting article. I was unaware that John Carmack worked on the code before it was released.

whatshisface 6y ago

Models are supposed to be based on first-principles reasoning. A "model" that's not designed to capture some aspect of reality is called a computer game...

SiempreViernes 6y ago

The Imperial model gives reproducible results, so most of the noise was professional programmers being salty about the fact that ugly code can still be accurate code. The fact that it predicted things like 80% of the population becoming infected was simply unrelated to code quality.

benmller313 6y ago
Regardless of code quality, are these models really useful if, as you say, they are "accurate", but still make predictions wildly out of step with reality?
- whatshisface 6y ago
  The real imperial story is that there were two wrong parties: - The scientists who were not including enough detail in their math. - The drive-by developers who had no idea what did or didn't make scientific simulations work. Together they produced both an inaccurate estimate, and a github page full of uninformed speculation about what the problem might have been. ;)
  - SiempreViernes 6y ago
    Honestly I think the main problem was they choose overly pessimistic estimates, or maybe that their model is so expensive they could only investigate a few choices of the parameters.
- Quarrelsome 6y ago
  same way that a given implementation is better than none. Its a concrete implementation that serves as an anchor. Its a fixed point to discuss implementation issues around as well as working as a harness that others can build on/around.
- SiempreViernes 6y ago
  Usually, because they help narrow down the sources of error. If you've tested your code so you're sure the various bits work as intended, any remaining somewhat crazy results are down to things you don't understand properly. A somewhat validated code allows you to do by computational experiments what you can't by real life experiments. Very handy if one of the common end states in the experiments is "death".
plantain 6y ago
Wasn't it more professional programmers being salty that policy decisions were being made by code with absolutely no tests or peer review?
- SiempreViernes 6y ago
  Code with no _unit tests_ or _code reviews_ possibly, it was obviously tested by the scientists that it made somewhat sensible and self-consistent predictions. In addition, predictions by earlier versions of the code had been peer reviewed, which is about as good as it gets outside of computer science. The report itself wasn't formally peer reviewed either, though in the crisis mentality of March this year not many people had the patience for peer review. Frankly, most still don't if you consider the amount of news about unreviewed science that get published.
bart_spoon 6y ago
Reproducibility doesn't mean accurate or good, it means it can be recreated. Its a crucial first step, but a model can be reproducible and also totally worthless.
- SiempreViernes 6y ago
  Yeah, but as you can see in this thread the rock star programmers of HN only know enough epidemiology to complain about code formatting and improper RNG seeding.
COVardsStayHome 6y ago
The same model that predicted that Sweden will have 100 000 dead by the end of June? https://www.medrxiv.org/content/10.1101/2020.04.11.20062133v1 https://www.medrxiv.org/content/10.1101/2020.04.11.20062133v... The authors of this junk model should be Hanged, drawn and quartered.
Jabbles 6y ago
gives? perhaps it does now, but it was certainly non-determinstic when it was first released, which is also when it was used for policy decisions. See for example https://news.ycombinator.com/item?id=23099386 https://news.ycombinator.com/item?id=23099386 But how useful was the model? It showed that the virus would spread exponentially, but we knew that. Is there anything that it showed that we didn't know when they programmed it, with constants tuned to what they expected?
- cinntaile 6y ago
  That was the point. They used stochastic partial differential equations, they're by definition non-deterministic.
  - Jabbles 6y ago
    No, they should be reproducible if run with the same seed.

mmcwilliams 6y ago

An interesting article. I was unaware that John Carmack worked on the code before it was released.