3 ms·
There were two popular models, in rough agreement. https://publichealth.jhu.edu/2020/transparency-reproducibility-and-validation-of-covid-19-projection-models h
by eesmith 4y ago
There were two popular models, in rough agreement. https://publichealth.jhu.edu/2020/transparency-reproducibility-and-validation-of-covid-19-projection-models https://publichealth.jhu.edu/2020/transparency-reproducibili...
> Among the two popular models, we found that the ICL model is more transparent and reproducible compared to the IHME model. The former sometimes over-predicted future deaths while the latter clearly under-predicted post-peak deaths. Both models predicted the timing of peaks reasonably well using data until one week prior. The ICL model produced a much wider band of uncertainty for New York state, possibly because the pattern did not conform well with their internal training data used from European countries.
Nature (on 08 June 2020) reported "a computational neuroscientist has reported that he has independently rerun the simulation and reproduced its results. And other scientists have told Nature that they had already privately verified that the code is reproducible." - https://www.nature.com/articles/d41586-020-01685-y https://www.nature.com/articles/d41586-020-01685-y citing the CODECHECK certificate at https://zenodo.org/record/3865491 https://zenodo.org/record/3865491 from May 29, 2020.
You should also be asking why public health is so under-funded world-wide that many countries looked to the ICL model - which in turn was a repurposed influenza model - rather than use local expertise or be able to use better validated models.
> Running the same simulation with the same inputs produced wildly different results between runs.
As I recall, the replication issues were the normal issues related to parallelization and random number generation, and not things which greatly affected the results. There were a number of HN threads on the topic back in 2020. Eg, https://news.ycombinator.com/item?id=23212268 https://news.ycombinator.com/item?id=23212268 , which linked to a couple of those issues and their resolution.
> I can't think of better example of a situation where requiring replication would have altered the path of humanity as much as it would have here.
I can. The Reinhart-Rogoff spreadsheet error which was widely used as the excuse for "austerity" cuts, rather than the real reason of wanting to lower taxes on the rich. https://truthout.org/articles/excel-spreadsheet-error-lessons-from-the-reinhart-rogoff-controversy/ https://truthout.org/articles/excel-spreadsheet-error-lesson...
I wonder where that tax money could have been used. Perhaps into public health measures?