15 ms·

Show HN: Covid-19 Interactive Model

291 points by nbnoll 7y ago

gus_massa 7y ago
Someone notices in another thread that in this model the death number does not depend on the number of beds or ICUs.
- nbnoll 7y ago
  We are working on this! It originally started out just as a visual representation however now it factors into the model. Check out branch feat/hospital-bed-model
  - gus_massa 7y ago
    Nice to hear that, because one of the important factors is if the health system gets oversaturated. Edit: For someone else that is interested, the link to the repo in GitHub, is in the "About" section at the top. Edit 2: The About section has a lot of info. Perhaps you should copy part of that info to the main page. Edit 3: When the site loads, it should show the graph of the simulation with the default parameters.
    - nbnoll 7y ago
      I just wanted to reply that the finite ICU capacity was merged last night and is now on the main webpage.
- TheGallopedHigh 7y ago
  This can in a sense be captured in the virulence rate factor. Lower rate factor equals more care available.
  - gus_massa 7y ago
    The main problem is the number of people with very bad symptoms (for example that need ICU). If there are enough ICU, then the mortality rate is like 0.1%. If there are not enough, the mortality is like 5%. (The numbers are difficult to measure, so they are only estimations.) Now they have added the ICU overflow, that is very important for the mortality rate.
nbnoll 7y ago
Hi everyone, My name is Nicholas Noll, a computational biologist and co-developer of the above tool. As of now there are two other people actively working on this project: Richard Neher, Ivan Aksamentov, and I. Richard and I have a background in theoretical physics but we have been working on quantitative biology and epidemiology. Richard is also one of the founders of nextstrain.org which leverages phylogenetic analyses to track the global spread of many pathogens -- including SARS-CoV-2. The other person, Ivan, is a super talented software engineer who just recently joined our group. We started working on this a few weeks ago, in collaboration with Jan Albert and Roberk Dyrdak, as a helpful model for public health officials and hospitals to help predict the usage of local community hospitals, and explore the utility and immediate need for quarantine measures. We are very much still actively developing the tool, now with an eye to help global communities at large. If anyone is interested to help, we are trying to organize a road map and put it together on our Github page. I'll be around to answer questions and listen to feedback.
- gbacon 7y ago
  We’ve all seen the time series of diagnosed cases in Italy versus the U.S. in which the latter appears to track closely but with a time delay. Is it relevant that the U.S. population is some 5 to 6 times Italy’s? Is it meaningful to compare populations by cases per capita?
  - nbnoll 7y ago
    In theory, the exponential growth rate of the virus doesn't depend on the population size. It more strongly depends on the population structure - i.e. the average number of social contacts made in a day.
    - dboreham 7y ago
      While there's exponential growth yes, but further up the logistic curve it makes a difference (you can't kill more people than the population of any country).
      - 7y ago
guscost 7y ago
Shouldn't this tool allow for modifying the numbers in the "Severity assumptions" section, too? This uses the same estimate from the Imperial College model, which is basically an educated guess at this point: > A total of 72,314 patient records—44,672 (61.8%) confirmed cases, 16,186 (22.4%) suspected cases, 10,567 (14.6%) clinically diagnosed cases (Hubei Province only), and 889 asymptomatic cases (1.2%)—contributed data for the analysis. Another study guesses that 86% of cases are asymptomatic and/or otherwise undocumented: https://science.sciencemag.org/content/early/2020/03/13/science.abb3221 https://science.sciencemag.org/content/early/2020/03/13/scie...
- nbnoll 7y ago
  All numbers in the table should be editable by the user -- apart from the fatality percentages which are computed from the values given. We've been working on the design to make this more obvious at a glance.
  - guscost 7y ago
    Aha, thank you! Nice work putting this together, it's much easier to understand than a paper.
    - nbnoll 7y ago
      No worries. I think it's also important to note that the Imperial College study that's gotten a lot of attention recently is an agent-based model - i.e. they simulate actual individuals - which allow them to look at specific quarantine measures like school closures. You can accomplish an approximate version of this with our model by controlling the isolation column of this table. This allows one to selectively quarantine age groups quantitatively.
      - guscost 7y ago
        Cool - again, nice work!
llamataboot 7y ago
The biggest thing I've been trying to get friends to understand (from 2 weeks ago on the we have to shut things down train, and now on the opposite "look not interacting with any other human being for 2 months might not be the best course of action train") is the non-linearity of how infection rate responds to decreasing the amount of social contacts. ie; in most models you can get about a 50% gain with just a 25% reduction in social contact, after that it is diminishing returns all the way down to the bottom of trying to wring small population level effects out of reducing contacts from "very little to very very little" In my mind, people are still underestimating the effects of large gatherings (and by large here I even mean 100 people in a nightclub) and overestimating the effects of seeing one friend in your house that sees another one friend the next day etc. Don't want to get into an overall debate here about what level of distancing is appropriate or acceptable, would like to see an interactive model that does a good job showing how different types of distancing interventions have very different effects and that once you get to a certain point it gets very hard to get much more overall connection out of the system...
- listsfrin 7y ago
  There is official health institute report from Italy: https://www.bloomberg.com/news/articles/2020-03-18/99-of-those-who-died-from-virus-had-other-illness-italy-says https://www.bloomberg.com/news/articles/2020-03-18/99-of-tho... This is just a panic created on social media.
  - llamataboot 7y ago
    maybe examine the prevalence of all those conditions in the US?
    - runin2k1 7y ago
      "More than 99% of Italy’s coronavirus fatalities were people who suffered from previous medical conditions" What were the most common pre-existing conditions? "More than 75% had high blood pressure, about 35% had diabetes and a third suffered from heart disease." How common are those conditions in the United States? High Blood Pressure - 1 in 3 Diabetes - 9 in 100 Heart Disease - 1 in 10
billions 7y ago
I built Sneezemaps.com to correlate symptoms & confirmed cases US: https://sneezemap.com/?zone=eyJjZW50ZXIiOls0MC4yNzk1MjU2Njg4MTI5MSwtOTguODc2OTUzMTI1MDAwMDFdLCJ6b29tIjo1fQ https://sneezemap.com/?zone=eyJjZW50ZXIiOls0MC4yNzk1MjU2Njg4... Europe: https://sneezemap.com/?zone=eyJjZW50ZXIiOls0Ny42NjUzODczNTYzMjY1NCwxMC4zNzEwOTM3NTAwMDAwMDJdLCJ6b29tIjo1fQ https://sneezemap.com/?zone=eyJjZW50ZXIiOls0Ny42NjUzODczNTYz...
- oehpr 7y ago
  this site isn't working for me, I grant location access and it says I need to turn on location access to see the map. Doesn't seem to be working? Fails on Chromium and Firefox.
ouid 7y ago
I closed this after "based on data from China". That's almost as bad as basing it on US data.
- dang 7y ago
  Please don't post unsubstantive and/or flamebait comments to HN.
throwaway_pdp09 7y ago
Interface is a not great. I wanted to compare US model to what's actually happening (I've been graphing the US curves with public data). Top dropbox is Scenario. Default is custom. I pick 'country - no mitigation'. Box below labelled 'Population' (eh?) now says 'Germany'. I pick 'United States' from it. The box below that also is labelled 'population' but has the value with no digit separation to make it readable (shows 330000000, which I initially misread under by an order of magnitude) but it does have an up/down arrows beside it to increment/decrement it by 1 (double eh?). The labels are not clearly associated with the dropdows, so it's easy to get confused over whether they refer to the box above or below. The help buttons are (blue show not hide text I think. The text they do have is often unhelpful eg. "ICU/ICMO (est.)" ask for help on that and you get "Number of ICU/ICMO available in health care system". Could be worse, could be better. The text box that pops up when you hover over the produced graphs are hard to read - you've got light text on a white background. Also the larger numbers are hard to read without digit separators. Also, don't get clever with the box following the pointer, it's not slick, it jumps around and is distracting. The graph axis is very logarithmic which is hard to get a feel for. Also the top figure shows for me as 0000000 because the leading '1' is hidden. I guess it's for professional of this area, which I'm not, so I'll step back now. HTH anyway. Not intended as dismissive if it sounds that way.
jupp0r 7y ago
It would be really nice if the uncertainty of fatality and severity information we have right now could be incorporated and visualized. Current reports by the CDC and info from France and Spain suggests that the age distribution of severe/critical cases might look very different than data from China made us believe. On the other hand, we don't know enough about the prevalence of asymptomatic cases, which have huge impacts on spread and severity, potentially. It would be really nice to model the probability distributions for each of these and see the resulting probabilities of simulation outcomes.
- jansan 7y ago
  Also, the huge uncertainty regarding the ratio of symptomless/unreported cases should be visualized. Guesses are in the range from 30 to 90%.
- Lukas1994 7y ago
  I'm working on a tool that lets you work with uncertainty very easily. The model I came up with is very trivial compared to Nicholas' though: https://my.causal.app/models/1432 https://my.causal.app/models/1432 Would be awesome if someone could build a more legit model in Causal - happy to help :)
deleted 7y ago
[deleted]
reubens 7y ago
I’d be interested in helping out with the front end. Having some issues with display on my iPhone SE. Thanks for the awesome tool!
chisleu 7y ago
Does it scare anyone that the North / Fast setting for the USA puts the death count at ~1/2 reality for the past few days? Does this indicate that it spread faster than fast? This is going to be really really bad isn't it?
- spookthesunset 7y ago
  Consider that in the US testing has barely started and even then there are a ton pre-conditions to get tested. Any “scary growth curve” of any absolute number you see is almost certainly a reflection of how much tests are being done. More tests means more confirmed results which means a nice scary looking curve. If you start with a hypothesis that this virus is already in widespread circulation and possibly has been for a while (which is the simplest explanation, mind you), odds are good many people already had it or have it right now. Only way to prove/disprove if it is widespread is by testing a random sample of the entire population. Something we aren’t doing in the US.
  - jhncls 7y ago
    What's really disturbing and hard to doubt is the growing daily death count in a country as Italy [1] [1] https://www.worldometers.info/coronavirus/country/italy/#graph-deaths-daily https://www.worldometers.info/coronavirus/country/italy/#gra...
    - badfrog 7y ago
      It looks like the number of deaths per day hasn't really been growing in the past ~5 days. That's somewhat comforting.
artur_makly 7y ago
503 ERROR
altaaf_baatli 7y ago
The link is not working right now
robhunter 7y ago
Just a suggestion, but some general thoughts on what activities correspond to what levels of mitigation - eg. are schools closing at 80%, restaurants at 70%? What activities result in which %'s?
toddm 7y ago
Thank you for providing a very nice tool that is of use for all of us. Much appreciated!
deleted 7y ago
[deleted]
patrickbuckley 7y ago
Thank you for making such a great tool. Do you have any plans that will allow the death rate to be adjusted once ICU overflow kicks in? There is going to be a much higher death rate once we run out of ICU beds and it would be interesting to see how that changes things
titzer 7y ago
Why is it when I reduce the infectious period, the steepness of the infection curve increases? That seems wrong.
- gridspy 7y ago
  Possibly because you forgot to reduce the R factor. If you infect people over 1/2 the time, perhaps you need R to be 1/2 what it was before. I assume that the model is spreading the people you infect across the time you are infectious.
Ameo 7y ago
After reading about the implementation of this model and experimenting with different a while, I've made some observations about its behavior and especially about the values that have been chosen for its for its parameters which I feel are important to consider when interpreting its output. TL;DR: Many of the critical inputs to this simulation are based off of almost entirely unknown values, and relatively small changes to these inputs can swing the results of the simulation by orders of magnitude. The defaults that the website creators have selected seem pessimistic on average compared to even the source research which can have the effect of misleading users as to what the expected outcome of the pandemic is. Since under-reacting and over-reacting to this situation both have huge real-world ramifications, making assessments based off of even a fully valid and well-designed model can lead to incorrect decisions being made if its parameters are set unrealistically. ---- First of all, there are a TON of variables and many of them pull values from extremely thin data. Although it looks like the researchers didn't just make up any of their values, there are several instances of very impactful parameters being hand-waved: ## MITIGATION CURVE The "Mitigation" ratio curve, an unsurprisingly critical variable for the simulation, seems too high even for the "strong" mitigation preset. Even after looking extensively, I couldn't find any explanation or justification from the simulation's creators as to how they created their default curve. As this is possibly the most impactful parameter in the entire simulation, that just makes it more critical to correctly pick its value. Judging by the degree of dramatic measures that society has taken so far and the actions of people that I've observed, I'd personally estimate that ratio to be much lower than even the one included in the "strong" mitigation curve. Of course this is random guesstimation of my own here, but it goes to illustrate the massive changes in results that can stem from small changes to the input values they've chosen. After starting at default settings, setting population to USA, and setting mitigation factor to "strong", further adjusting the March 15 mitigation factor value down 10% from 60% to 50% (a value that seems extremely reasonable and even generous to me) causes the total number of cases to drop by nearly 66% holding all other values constant. Adjusting it to 40% reduces the total infections by another nearly 50% from there. Another paper that is cited by the simulation's authors writes that "[...] by taking drastic social distancing measures and policies of controlling the source of infection, with the tremendous joint efforts from the government, healthcare workers, and the people (Fig. 1), Rt was substantially reduced [from 3.58] to 0.32 in Wuhan after February 2, which was encouraging for the global efforts fighting against the Covid-19 outbreak using traditional non-pharmaceutical measures [...]" [4] This would seem to represent a mitigation ratio of 0.1 after just around a month. Clearly there are differences in the way that the pandemic was handled in China compared to how it's being handled now in the rest of the world, but China's ability to combat it that effectively seems to lend credence to the idea that mitigation ratios of less than 0.5 are possible if not already in place in the US and across the rest of the world. In any case, claiming nearly 80% of the transmissibility of this virus remaining at this point in time and maxing out at 60% as the "moderate" case seems dangerously exaggerated to the upside and potentially skews the results of the whole simulation due to how critical this one variable is in determining the output. ## ANNUAL AVERAGE R0 Another foundational variable is R0: The number of additional infected individuals per infection. The linked research paper states "the early human-to-human transmission of 2019-nCoV was characterized by values of R0 around 2.2 (median value, with 90% high density interval: 1.4–3.8)" [2] Although the distribution from the research paper has a long tail towards higher values, setting a value of 2.7 for "Moderate/North" feels somewhat disingenuous given that the mean is 2.2. Of course other research papers list a wide range of different values for this variable, so perhaps expanding the range of the presets would be a better option. In any case, this variable is dominated by the mitigation factor in cases of high mitigation, so its precise value may not matter as much in those situations. ## SEASONAL FORCING The "seasonal forcing" factor varies from 0 to 0.2 in all of their "Epidemiology" presets. On their about page, the example they provide in their graph seems to have a value of ~0.6, but that may just be an illustration using non-realistic values for visual effect. [1] One other thing to note is that their implementation uses the selected R0 as a mean value for their function, meaning that at the peak month the true R0 is `(1 + <seasonal forcing>) * R0`. This doesn't agree with the data from the research paper, which estimated their R0 value using data from the most infectious period (winter). Assuming a seasonal forcing factor of 0.2, that means The R0 values they provide are actually 20% inflated in January on top of their already high values. Of course, their very choice of a cosine wave to model that impact seems largely unfounded, but of course given a lack of data pointing to a more accurate option it's as good as any. That being said, even a small change to that function could have massive impacts on the simulation's results. ## ICU OVERFLOW In the most pessimistic scenarios, their implementation of additional fatalities caused by ICU overflow fail to take into account potential measures such as emergency hospitals, things like the hospital ship being sent over to New York by the navy [3], and temporary increases to hospital capacity. In most cases, the peak of ICU overflow doesn't occur until months down the line, giving a lot of time to scrape up resources to increase capacity. Perhaps providing an additional value to adjust the hospital capacity over time would be useful for accurately representing the impact of ICU overflow. ## OTHER VARIABLES + NOTES - "Imports" is held constant through the entire simulation. It seems unlikely that it would remain this way; setting this as a curve instead would make more sense in my opinion. Also, I couldn't find any justification or explanation for the values they picked for that parameter. - Improvements to treatment causing ICU/hospital stay time to go down aren't accounted for. Of course there's no guarantee that it will change significantly as the pandemic progresses (or even that it will not get worse due to some mutation of the virus or other circumstance), but recent research and experimentation seems to already be making progress in creating more effective treatments for the virus. - Due to how incredibly vast the system this model models is, there are an uncountable number of possible external events that could, in some situations, invalidate the whole thing entirely. At least mentioning the possibility of things such as a vaccine being released in the coming months, mutations with developed antiviral drug resistance, etc. seems prudent to me. ---- NOTE: I'm not a trained statistician or epidemiologist - I just have experience working with data and compute models. I'd appreciate any feedback about the accuracy of my comments here or expansions/refutations of my thinking. [1] Link from about page: https://neherlab.org/covid19/assets/seasonal_illustration.151cec1.svg; https://neherlab.org/covid19/assets/seasonal_illustration.15... Screenshot at time of writing: https://ameo.link/u/7pk.png https://ameo.link/u/7pk.png [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7001239/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7001239/ [3] https://www.nbcnewyork.com/news/local/navy-medical-ship-coming-to-relieve-nyc-hospitals-amid-virus-was-last-in-city-after-9-11/2333767/ https://www.nbcnewyork.com/news/local/navy-medical-ship-comi... [4] https://www.medrxiv.org/content/10.1101/2020.03.03.20030593v1.full.pdf https://www.medrxiv.org/content/10.1101/2020.03.03.20030593v...

291 points by nbnoll 7y ago

gus_massa 7y ago

Someone notices in another thread that in this model the death number does not depend on the number of beds or ICUs.

nbnoll 7y ago
We are working on this! It originally started out just as a visual representation however now it factors into the model. Check out branch feat/hospital-bed-model
- gus_massa 7y ago
  Nice to hear that, because one of the important factors is if the health system gets oversaturated. Edit: For someone else that is interested, the link to the repo in GitHub, is in the "About" section at the top. Edit 2: The About section has a lot of info. Perhaps you should copy part of that info to the main page. Edit 3: When the site loads, it should show the graph of the simulation with the default parameters.
  - nbnoll 7y ago
    I just wanted to reply that the finite ICU capacity was merged last night and is now on the main webpage.
TheGallopedHigh 7y ago
This can in a sense be captured in the virulence rate factor. Lower rate factor equals more care available.
- gus_massa 7y ago
  The main problem is the number of people with very bad symptoms (for example that need ICU). If there are enough ICU, then the mortality rate is like 0.1%. If there are not enough, the mortality is like 5%. (The numbers are difficult to measure, so they are only estimations.) Now they have added the ICU overflow, that is very important for the mortality rate.

nbnoll 7y ago

Hi everyone, My name is Nicholas Noll, a computational biologist and co-developer of the above tool. As of now there are two other people actively working on this project: Richard Neher, Ivan Aksamentov, and I. Richard and I have a background in theoretical physics but we have been working on quantitative biology and epidemiology. Richard is also one of the founders of nextstrain.org which leverages phylogenetic analyses to track the global spread of many pathogens -- including SARS-CoV-2. The other person, Ivan, is a super talented software engineer who just recently joined our group. We started working on this a few weeks ago, in collaboration with Jan Albert and Roberk Dyrdak, as a helpful model for public health officials and hospitals to help predict the usage of local community hospitals, and explore the utility and immediate need for quarantine measures. We are very much still actively developing the tool, now with an eye to help global communities at large. If anyone is interested to help, we are trying to organize a road map and put it together on our Github page. I'll be around to answer questions and listen to feedback.

gbacon 7y ago
We’ve all seen the time series of diagnosed cases in Italy versus the U.S. in which the latter appears to track closely but with a time delay. Is it relevant that the U.S. population is some 5 to 6 times Italy’s? Is it meaningful to compare populations by cases per capita?
- nbnoll 7y ago
  In theory, the exponential growth rate of the virus doesn't depend on the population size. It more strongly depends on the population structure - i.e. the average number of social contacts made in a day.
  - dboreham 7y ago
    While there's exponential growth yes, but further up the logistic curve it makes a difference (you can't kill more people than the population of any country).
    - 7y ago

guscost 7y ago

Shouldn't this tool allow for modifying the numbers in the "Severity assumptions" section, too? This uses the same estimate from the Imperial College model, which is basically an educated guess at this point: > A total of 72,314 patient records—44,672 (61.8%) confirmed cases, 16,186 (22.4%) suspected cases, 10,567 (14.6%) clinically diagnosed cases (Hubei Province only), and 889 asymptomatic cases (1.2%)—contributed data for the analysis. Another study guesses that 86% of cases are asymptomatic and/or otherwise undocumented: https://science.sciencemag.org/content/early/2020/03/13/science.abb3221 https://science.sciencemag.org/content/early/2020/03/13/scie...

nbnoll 7y ago
All numbers in the table should be editable by the user -- apart from the fatality percentages which are computed from the values given. We've been working on the design to make this more obvious at a glance.
- guscost 7y ago
  Aha, thank you! Nice work putting this together, it's much easier to understand than a paper.
  - nbnoll 7y ago
    No worries. I think it's also important to note that the Imperial College study that's gotten a lot of attention recently is an agent-based model - i.e. they simulate actual individuals - which allow them to look at specific quarantine measures like school closures. You can accomplish an approximate version of this with our model by controlling the isolation column of this table. This allows one to selectively quarantine age groups quantitatively.
    - guscost 7y ago
      Cool - again, nice work!

llamataboot 7y ago

The biggest thing I've been trying to get friends to understand (from 2 weeks ago on the we have to shut things down train, and now on the opposite "look not interacting with any other human being for 2 months might not be the best course of action train") is the non-linearity of how infection rate responds to decreasing the amount of social contacts. ie; in most models you can get about a 50% gain with just a 25% reduction in social contact, after that it is diminishing returns all the way down to the bottom of trying to wring small population level effects out of reducing contacts from "very little to very very little" In my mind, people are still underestimating the effects of large gatherings (and by large here I even mean 100 people in a nightclub) and overestimating the effects of seeing one friend in your house that sees another one friend the next day etc. Don't want to get into an overall debate here about what level of distancing is appropriate or acceptable, would like to see an interactive model that does a good job showing how different types of distancing interventions have very different effects and that once you get to a certain point it gets very hard to get much more overall connection out of the system...

listsfrin 7y ago
There is official health institute report from Italy: https://www.bloomberg.com/news/articles/2020-03-18/99-of-those-who-died-from-virus-had-other-illness-italy-says https://www.bloomberg.com/news/articles/2020-03-18/99-of-tho... This is just a panic created on social media.
- llamataboot 7y ago
  maybe examine the prevalence of all those conditions in the US?
  - runin2k1 7y ago
    "More than 99% of Italy’s coronavirus fatalities were people who suffered from previous medical conditions" What were the most common pre-existing conditions? "More than 75% had high blood pressure, about 35% had diabetes and a third suffered from heart disease." How common are those conditions in the United States? High Blood Pressure - 1 in 3 Diabetes - 9 in 100 Heart Disease - 1 in 10

billions 7y ago

I built Sneezemaps.com to correlate symptoms & confirmed cases US: https://sneezemap.com/?zone=eyJjZW50ZXIiOls0MC4yNzk1MjU2Njg4MTI5MSwtOTguODc2OTUzMTI1MDAwMDFdLCJ6b29tIjo1fQ https://sneezemap.com/?zone=eyJjZW50ZXIiOls0MC4yNzk1MjU2Njg4... Europe: https://sneezemap.com/?zone=eyJjZW50ZXIiOls0Ny42NjUzODczNTYzMjY1NCwxMC4zNzEwOTM3NTAwMDAwMDJdLCJ6b29tIjo1fQ https://sneezemap.com/?zone=eyJjZW50ZXIiOls0Ny42NjUzODczNTYz...

oehpr 7y ago
this site isn't working for me, I grant location access and it says I need to turn on location access to see the map. Doesn't seem to be working? Fails on Chromium and Firefox.

ouid 7y ago

I closed this after "based on data from China". That's almost as bad as basing it on US data.

dang 7y ago
Please don't post unsubstantive and/or flamebait comments to HN.

throwaway_pdp09 7y ago

Interface is a not great. I wanted to compare US model to what's actually happening (I've been graphing the US curves with public data). Top dropbox is Scenario. Default is custom. I pick 'country - no mitigation'. Box below labelled 'Population' (eh?) now says 'Germany'. I pick 'United States' from it. The box below that also is labelled 'population' but has the value with no digit separation to make it readable (shows 330000000, which I initially misread under by an order of magnitude) but it does have an up/down arrows beside it to increment/decrement it by 1 (double eh?). The labels are not clearly associated with the dropdows, so it's easy to get confused over whether they refer to the box above or below. The help buttons are (blue show not hide text I think. The text they do have is often unhelpful eg. "ICU/ICMO (est.)" ask for help on that and you get "Number of ICU/ICMO available in health care system". Could be worse, could be better. The text box that pops up when you hover over the produced graphs are hard to read - you've got light text on a white background. Also the larger numbers are hard to read without digit separators. Also, don't get clever with the box following the pointer, it's not slick, it jumps around and is distracting. The graph axis is very logarithmic which is hard to get a feel for. Also the top figure shows for me as 0000000 because the leading '1' is hidden. I guess it's for professional of this area, which I'm not, so I'll step back now. HTH anyway. Not intended as dismissive if it sounds that way.

jupp0r 7y ago

It would be really nice if the uncertainty of fatality and severity information we have right now could be incorporated and visualized. Current reports by the CDC and info from France and Spain suggests that the age distribution of severe/critical cases might look very different than data from China made us believe. On the other hand, we don't know enough about the prevalence of asymptomatic cases, which have huge impacts on spread and severity, potentially. It would be really nice to model the probability distributions for each of these and see the resulting probabilities of simulation outcomes.

jansan 7y ago
Also, the huge uncertainty regarding the ratio of symptomless/unreported cases should be visualized. Guesses are in the range from 30 to 90%.
Lukas1994 7y ago
I'm working on a tool that lets you work with uncertainty very easily. The model I came up with is very trivial compared to Nicholas' though: https://my.causal.app/models/1432 https://my.causal.app/models/1432 Would be awesome if someone could build a more legit model in Causal - happy to help :)

deleted 7y ago

[deleted]

reubens 7y ago

I’d be interested in helping out with the front end. Having some issues with display on my iPhone SE. Thanks for the awesome tool!

chisleu 7y ago

Does it scare anyone that the North / Fast setting for the USA puts the death count at ~1/2 reality for the past few days? Does this indicate that it spread faster than fast? This is going to be really really bad isn't it?

spookthesunset 7y ago
Consider that in the US testing has barely started and even then there are a ton pre-conditions to get tested. Any “scary growth curve” of any absolute number you see is almost certainly a reflection of how much tests are being done. More tests means more confirmed results which means a nice scary looking curve. If you start with a hypothesis that this virus is already in widespread circulation and possibly has been for a while (which is the simplest explanation, mind you), odds are good many people already had it or have it right now. Only way to prove/disprove if it is widespread is by testing a random sample of the entire population. Something we aren’t doing in the US.
- jhncls 7y ago
  What's really disturbing and hard to doubt is the growing daily death count in a country as Italy [1] [1] https://www.worldometers.info/coronavirus/country/italy/#graph-deaths-daily https://www.worldometers.info/coronavirus/country/italy/#gra...
  - badfrog 7y ago
    It looks like the number of deaths per day hasn't really been growing in the past ~5 days. That's somewhat comforting.

artur_makly 7y ago

503 ERROR

altaaf_baatli 7y ago

The link is not working right now

robhunter 7y ago

Just a suggestion, but some general thoughts on what activities correspond to what levels of mitigation - eg. are schools closing at 80%, restaurants at 70%? What activities result in which %'s?

toddm 7y ago

Thank you for providing a very nice tool that is of use for all of us. Much appreciated!

deleted 7y ago

[deleted]

patrickbuckley 7y ago

Thank you for making such a great tool. Do you have any plans that will allow the death rate to be adjusted once ICU overflow kicks in? There is going to be a much higher death rate once we run out of ICU beds and it would be interesting to see how that changes things

titzer 7y ago

Why is it when I reduce the infectious period, the steepness of the infection curve increases? That seems wrong.

gridspy 7y ago
Possibly because you forgot to reduce the R factor. If you infect people over 1/2 the time, perhaps you need R to be 1/2 what it was before. I assume that the model is spreading the people you infect across the time you are infectious.

Ameo 7y ago

After reading about the implementation of this model and experimenting with different a while, I've made some observations about its behavior and especially about the values that have been chosen for its for its parameters which I feel are important to consider when interpreting its output. TL;DR: Many of the critical inputs to this simulation are based off of almost entirely unknown values, and relatively small changes to these inputs can swing the results of the simulation by orders of magnitude. The defaults that the website creators have selected seem pessimistic on average compared to even the source research which can have the effect of misleading users as to what the expected outcome of the pandemic is. Since under-reacting and over-reacting to this situation both have huge real-world ramifications, making assessments based off of even a fully valid and well-designed model can lead to incorrect decisions being made if its parameters are set unrealistically. ---- First of all, there are a TON of variables and many of them pull values from extremely thin data. Although it looks like the researchers didn't just make up any of their values, there are several instances of very impactful parameters being hand-waved: ## MITIGATION CURVE The "Mitigation" ratio curve, an unsurprisingly critical variable for the simulation, seems too high even for the "strong" mitigation preset. Even after looking extensively, I couldn't find any explanation or justification from the simulation's creators as to how they created their default curve. As this is possibly the most impactful parameter in the entire simulation, that just makes it more critical to correctly pick its value. Judging by the degree of dramatic measures that society has taken so far and the actions of people that I've observed, I'd personally estimate that ratio to be much lower than even the one included in the "strong" mitigation curve. Of course this is random guesstimation of my own here, but it goes to illustrate the massive changes in results that can stem from small changes to the input values they've chosen. After starting at default settings, setting population to USA, and setting mitigation factor to "strong", further adjusting the March 15 mitigation factor value down 10% from 60% to 50% (a value that seems extremely reasonable and even generous to me) causes the total number of cases to drop by nearly 66% holding all other values constant. Adjusting it to 40% reduces the total infections by another nearly 50% from there. Another paper that is cited by the simulation's authors writes that "[...] by taking drastic social distancing measures and policies of controlling the source of infection, with the tremendous joint efforts from the government, healthcare workers, and the people (Fig. 1), Rt was substantially reduced [from 3.58] to 0.32 in Wuhan after February 2, which was encouraging for the global efforts fighting against the Covid-19 outbreak using traditional non-pharmaceutical measures [...]" [4] This would seem to represent a mitigation ratio of 0.1 after just around a month. Clearly there are differences in the way that the pandemic was handled in China compared to how it's being handled now in the rest of the world, but China's ability to combat it that effectively seems to lend credence to the idea that mitigation ratios of less than 0.5 are possible if not already in place in the US and across the rest of the world. In any case, claiming nearly 80% of the transmissibility of this virus remaining at this point in time and maxing out at 60% as the "moderate" case seems dangerously exaggerated to the upside and potentially skews the results of the whole simulation due to how critical this one variable is in determining the output. ## ANNUAL AVERAGE R0 Another foundational variable is R0: The number of additional infected individuals per infection. The linked research paper states "the early human-to-human transmission of 2019-nCoV was characterized by values of R0 around 2.2 (median value, with 90% high density interval: 1.4–3.8)" [2] Although the distribution from the research paper has a long tail towards higher values, setting a value of 2.7 for "Moderate/North" feels somewhat disingenuous given that the mean is 2.2. Of course other research papers list a wide range of different values for this variable, so perhaps expanding the range of the presets would be a better option. In any case, this variable is dominated by the mitigation factor in cases of high mitigation, so its precise value may not matter as much in those situations. ## SEASONAL FORCING The "seasonal forcing" factor varies from 0 to 0.2 in all of their "Epidemiology" presets. On their about page, the example they provide in their graph seems to have a value of ~0.6, but that may just be an illustration using non-realistic values for visual effect. [1] One other thing to note is that their implementation uses the selected R0 as a mean value for their function, meaning that at the peak month the true R0 is `(1 + <seasonal forcing>) * R0`. This doesn't agree with the data from the research paper, which estimated their R0 value using data from the most infectious period (winter). Assuming a seasonal forcing factor of 0.2, that means The R0 values they provide are actually 20% inflated in January on top of their already high values. Of course, their very choice of a cosine wave to model that impact seems largely unfounded, but of course given a lack of data pointing to a more accurate option it's as good as any. That being said, even a small change to that function could have massive impacts on the simulation's results. ## ICU OVERFLOW In the most pessimistic scenarios, their implementation of additional fatalities caused by ICU overflow fail to take into account potential measures such as emergency hospitals, things like the hospital ship being sent over to New York by the navy [3], and temporary increases to hospital capacity. In most cases, the peak of ICU overflow doesn't occur until months down the line, giving a lot of time to scrape up resources to increase capacity. Perhaps providing an additional value to adjust the hospital capacity over time would be useful for accurately representing the impact of ICU overflow. ## OTHER VARIABLES + NOTES - "Imports" is held constant through the entire simulation. It seems unlikely that it would remain this way; setting this as a curve instead would make more sense in my opinion. Also, I couldn't find any justification or explanation for the values they picked for that parameter. - Improvements to treatment causing ICU/hospital stay time to go down aren't accounted for. Of course there's no guarantee that it will change significantly as the pandemic progresses (or even that it will not get worse due to some mutation of the virus or other circumstance), but recent research and experimentation seems to already be making progress in creating more effective treatments for the virus. - Due to how incredibly vast the system this model models is, there are an uncountable number of possible external events that could, in some situations, invalidate the whole thing entirely. At least mentioning the possibility of things such as a vaccine being released in the coming months, mutations with developed antiviral drug resistance, etc. seems prudent to me. ---- NOTE: I'm not a trained statistician or epidemiologist - I just have experience working with data and compute models. I'd appreciate any feedback about the accuracy of my comments here or expansions/refutations of my thinking. [1] Link from about page: https://neherlab.org/covid19/assets/seasonal_illustration.151cec1.svg; https://neherlab.org/covid19/assets/seasonal_illustration.15... Screenshot at time of writing: https://ameo.link/u/7pk.png https://ameo.link/u/7pk.png [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7001239/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7001239/ [3] https://www.nbcnewyork.com/news/local/navy-medical-ship-coming-to-relieve-nyc-hospitals-amid-virus-was-last-in-city-after-9-11/2333767/ https://www.nbcnewyork.com/news/local/navy-medical-ship-comi... [4] https://www.medrxiv.org/content/10.1101/2020.03.03.20030593v1.full.pdf https://www.medrxiv.org/content/10.1101/2020.03.03.20030593v...