5 ms·

> We need more studies to gather data on these asymptomatic cases if we want to reopen the economy soon. I don't understand what's so hard about measuring popu

> We need more studies to gather data on these asymptomatic cases if we want to reopen the economy soon. I don't understand what's so hard about measuring population infection rate. Assuming the population infection rate is between 1-10%, we would only need to do around ~500 randomized tests to achieve a 95% confidence interval of +/- 1%. For example, let's say we tested 500 random NYC residents for COVID and found that 10 were positive -- a 2% infection rate. The standard error (binomial approximation) for this sample is 0.6%. So, by doing a mere 500 randomized tests, we have a 95% confidence interval of 0.8%-3.2%. Given that overall population infection rate is so important in planning for re-opening, why are we not doing randomized testing in hotspots like NY on a regular basis? Am I missing something?

DenisM 6y ago
False positive rate of some antibody tests is 9%, which makes any effect size below that invisible.
- deleted 6y ago
  [deleted]
- duxup 6y ago
  Jebus, 9% false positive ... is that even useful?
  - arcticfox 6y ago
    I mean, it's not great, but imagine if it was so cheap and fast we could test the entire population on a consistent basis. In a hypothetical scenario of 1% of the population actually having COVID and 9% testing false positive, you could ask all 10 positive results to self-quarantine and that'd probably be a pretty effective way of shutting down the virus without asking the whole population to stay home. Once the virus spreads more, it gets even more reasonable. (If it gets crushed more and there are very few cases, it does get a little extreme to call it useful though).
    - DenisM 6y ago
      You need to also take into account the false positive too (1% iirc). Under your priors a group of 100,000 people will have 1000 infected and 99,000 not infected. Specifically: 99,0000.09 = 8,910 false positives 99,0000.91 = 90,090 true negatives 1,0000.99 = 990 true positive 1,0000.01 = 10 false negative So going through your plan will isolate 8,910 + 990 = 9900 people (9.9% of the population), catching 990 actual cases and letting loose 10 cases. In other words we can isolate 10% of the population and reduce the number of carriers by 990:10 ratio. This seems more effective than the current 100% isolation. All we need is the tests now.
      - rjvs 6y ago
        > All we need is the tests now. Plus people to then do what they are told...
  - DenisM 6y ago
    Great question, and the one I struggled with a lot. When I finally saw the light it was like find the glasses I have misplaced three days ago. Here: 1. Efficacy of a test can only be judged against specific priors, not any possible circumstance. 2. Likewise, the goal of a test is to assist in making a particular decision, not all possible decisions. Specifically 9% false positive rate is not useful for testing general population where effect size is expected to be on the order of 1-10%. However it is very useful in testing groups where expected effect size is much larger for example "all symptomatic people who came to hospital" at 50% or "all contacts of a known case" at 20%. The numbers are made up to illustrate the math. Importantly our goal is not to detect all infected people ending the epidemic in XYZ days flat, it is to reduce the viral spread factor below 0.5, halving the epidemic every XYZ days. Thinking in absolutes is counter-productive. Bearing all this in mind, the test can be useful. For example let's take a pool of people who are symptomatic, and say we expect 50% were in fact infected. The false positives (9% of the healthy 50% == 4.5%) will be outnumbered by true positives (* 99% of the infected 50% == 49%). So now you're looking at 49% vs 4.5%, a respectable 11:1 accuracy under the given priors. Not bad, for some applications. And here's one good application: test a group of people from the pool of the currently symptomatic, quarantine them for two weeks, then release them into the wild without self-isolation rules. They will all be healthy due to quarantine, and 10/11 will be immune. If we keep releasing 91% immune groups of people into the general population the virus will die off. * the antibody test I was referring to has 99% true positive and 9% false positive rates.
    - duxup 6y ago
      Thank you, yeah at large scale I could see how that might help.
- malpighien 6y ago
  It would be the test using QPCR to amplify the virus genomic material from samples collection. That test is a lot more specific though there can be issues with how the samples were obtained.
- smallgovt 6y ago
  Ah, I see. Is it true that these tests are a work in progress and the false positive rate will drop in the coming weeks/months? If so, it seems like we should start doing random sampling and saving the samples for future testing (so we can see how infection rate trended). This could be invaluable data that we're losing.
bgorman 6y ago
The thing you are missing is world governments want to milk this crisis, not act objectively. There is simply no other explanation.
- ben_w 6y ago
  Another explanation is that the exact infection level isn’t important when the observed all-causes fatality rate is double [0] its seasonal average and everyone is therefore busy firefighting the immediate and obvious problems. [0] varies by region, but see for example this graph of London: https://pbs.twimg.com/media/EVlO-28XQAA30xu?format=jpg&name=large https://pbs.twimg.com/media/EVlO-28XQAA30xu?format=jpg&name=...
  - bgorman 6y ago
    Don't take my word for it. Here is an article by a Stanford epidemiologist calling for random testing a month ago: https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable-data/ https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-a... We need data to act objectively. As far as I can tell US state governments are doing all they can to prevent random data from being released.
    - stetrain 6y ago
      What's the motivation? I see plenty of motivation in some states to try to minimize the potential magnitude of the problem, or avoid "bad" numbers of infection rates. I don't see a motivation for "milking" the crisis. The current actions by governments are restricting economic activity and revenue streams. Many states only make money on sales tax and that's probably not going super well right now. I agree that there is hesitation to do widespread testing, but I don't know which government would intentionally cause a recession using this virus as an excuse.
      - mrguyorama 6y ago
        The parent is parroting a conservative talking point: That the current situation is being pushed by democrats to hurt the economy and thus Trump
irishcoffee 6y ago
I'm hopeful the results of this shed some light on things: https://news.usc.edu/168497/antibody-testing-covid-19-pandemic-usc-los-angeles-county/ https://news.usc.edu/168497/antibody-testing-covid-19-pandem...
stefco_ 6y ago
There's good reason to suspect strong regional variation, which means that finding a representative sample of 500 is hard. But that would still be technically possible (look at South Korea and China); we clearly lack to political will to do proper testing in USA.
deleted 6y ago
[deleted]
dr_dshiv 6y ago
Amen. The whole scenario seems like a bad statistics lesson. Happy to change my mind -- but there simply hasn't been any effort to use population testing in this way -- which is one of the only useful forms of testing. Otherwise, why test people in the hospital? It doesn't change treatment. Finding asymptomatics is actually useful -- and random sampling seems critical for understanding whether we are simply fucked or actually fubar'd.
- snewman 6y ago
  Agreed generally, but my understanding is that testing people in the hospital does have value. Just for one example, if you're negative, then you can be placed in a non-Covid wing (otherwise you're liable to catch the coronavirus on top of whatever flu or whatever you actually have), staff don't need to use precious PPE when treating you, etc.
riazrizvi 6y ago
Well we need to measure if people have it and if people have had it. The have it test is currently $3000 a pop, and the had it test is hitting accuracy road blocks [1]. [1] https://www.wsj.com/articles/health-authorities-roll-out-new-coronavirus-tests-to-gauge-infections-spread-11586511004 https://www.wsj.com/articles/health-authorities-roll-out-new...

DenisM 6y ago

False positive rate of some antibody tests is 9%, which makes any effect size below that invisible.

deleted 6y ago
[deleted]
duxup 6y ago
Jebus, 9% false positive ... is that even useful?
- arcticfox 6y ago
  I mean, it's not great, but imagine if it was so cheap and fast we could test the entire population on a consistent basis. In a hypothetical scenario of 1% of the population actually having COVID and 9% testing false positive, you could ask all 10 positive results to self-quarantine and that'd probably be a pretty effective way of shutting down the virus without asking the whole population to stay home. Once the virus spreads more, it gets even more reasonable. (If it gets crushed more and there are very few cases, it does get a little extreme to call it useful though).
  - DenisM 6y ago
    You need to also take into account the false positive too (1% iirc). Under your priors a group of 100,000 people will have 1000 infected and 99,000 not infected. Specifically: 99,0000.09 = 8,910 false positives 99,0000.91 = 90,090 true negatives 1,0000.99 = 990 true positive 1,0000.01 = 10 false negative So going through your plan will isolate 8,910 + 990 = 9900 people (9.9% of the population), catching 990 actual cases and letting loose 10 cases. In other words we can isolate 10% of the population and reduce the number of carriers by 990:10 ratio. This seems more effective than the current 100% isolation. All we need is the tests now.
    - rjvs 6y ago
      > All we need is the tests now. Plus people to then do what they are told...
- DenisM 6y ago
  Great question, and the one I struggled with a lot. When I finally saw the light it was like find the glasses I have misplaced three days ago. Here: 1. Efficacy of a test can only be judged against specific priors, not any possible circumstance. 2. Likewise, the goal of a test is to assist in making a particular decision, not all possible decisions. Specifically 9% false positive rate is not useful for testing general population where effect size is expected to be on the order of 1-10%. However it is very useful in testing groups where expected effect size is much larger for example "all symptomatic people who came to hospital" at 50% or "all contacts of a known case" at 20%. The numbers are made up to illustrate the math. Importantly our goal is not to detect all infected people ending the epidemic in XYZ days flat, it is to reduce the viral spread factor below 0.5, halving the epidemic every XYZ days. Thinking in absolutes is counter-productive. Bearing all this in mind, the test can be useful. For example let's take a pool of people who are symptomatic, and say we expect 50% were in fact infected. The false positives (9% of the healthy 50% == 4.5%) will be outnumbered by true positives (* 99% of the infected 50% == 49%). So now you're looking at 49% vs 4.5%, a respectable 11:1 accuracy under the given priors. Not bad, for some applications. And here's one good application: test a group of people from the pool of the currently symptomatic, quarantine them for two weeks, then release them into the wild without self-isolation rules. They will all be healthy due to quarantine, and 10/11 will be immune. If we keep releasing 91% immune groups of people into the general population the virus will die off. * the antibody test I was referring to has 99% true positive and 9% false positive rates.
  - duxup 6y ago
    Thank you, yeah at large scale I could see how that might help.
malpighien 6y ago
It would be the test using QPCR to amplify the virus genomic material from samples collection. That test is a lot more specific though there can be issues with how the samples were obtained.
smallgovt 6y ago
Ah, I see. Is it true that these tests are a work in progress and the false positive rate will drop in the coming weeks/months? If so, it seems like we should start doing random sampling and saving the samples for future testing (so we can see how infection rate trended). This could be invaluable data that we're losing.

bgorman 6y ago

The thing you are missing is world governments want to milk this crisis, not act objectively. There is simply no other explanation.

ben_w 6y ago
Another explanation is that the exact infection level isn’t important when the observed all-causes fatality rate is double [0] its seasonal average and everyone is therefore busy firefighting the immediate and obvious problems. [0] varies by region, but see for example this graph of London: https://pbs.twimg.com/media/EVlO-28XQAA30xu?format=jpg&name=large https://pbs.twimg.com/media/EVlO-28XQAA30xu?format=jpg&name=...
- bgorman 6y ago
  Don't take my word for it. Here is an article by a Stanford epidemiologist calling for random testing a month ago: https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-as-the-coronavirus-pandemic-takes-hold-we-are-making-decisions-without-reliable-data/ https://www.statnews.com/2020/03/17/a-fiasco-in-the-making-a... We need data to act objectively. As far as I can tell US state governments are doing all they can to prevent random data from being released.
  - stetrain 6y ago
    What's the motivation? I see plenty of motivation in some states to try to minimize the potential magnitude of the problem, or avoid "bad" numbers of infection rates. I don't see a motivation for "milking" the crisis. The current actions by governments are restricting economic activity and revenue streams. Many states only make money on sales tax and that's probably not going super well right now. I agree that there is hesitation to do widespread testing, but I don't know which government would intentionally cause a recession using this virus as an excuse.
    - mrguyorama 6y ago
      The parent is parroting a conservative talking point: That the current situation is being pushed by democrats to hurt the economy and thus Trump

irishcoffee 6y ago

I'm hopeful the results of this shed some light on things: https://news.usc.edu/168497/antibody-testing-covid-19-pandemic-usc-los-angeles-county/ https://news.usc.edu/168497/antibody-testing-covid-19-pandem...

stefco_ 6y ago

There's good reason to suspect strong regional variation, which means that finding a representative sample of 500 is hard. But that would still be technically possible (look at South Korea and China); we clearly lack to political will to do proper testing in USA.

deleted 6y ago

[deleted]

dr_dshiv 6y ago

Amen. The whole scenario seems like a bad statistics lesson. Happy to change my mind -- but there simply hasn't been any effort to use population testing in this way -- which is one of the only useful forms of testing. Otherwise, why test people in the hospital? It doesn't change treatment. Finding asymptomatics is actually useful -- and random sampling seems critical for understanding whether we are simply fucked or actually fubar'd.

snewman 6y ago
Agreed generally, but my understanding is that testing people in the hospital does have value. Just for one example, if you're negative, then you can be placed in a non-Covid wing (otherwise you're liable to catch the coronavirus on top of whatever flu or whatever you actually have), staff don't need to use precious PPE when treating you, etc.

riazrizvi 6y ago

Well we need to measure if people have it and if people have had it. The have it test is currently $3000 a pop, and the had it test is hitting accuracy road blocks [1]. [1] https://www.wsj.com/articles/health-authorities-roll-out-new-coronavirus-tests-to-gauge-infections-spread-11586511004 https://www.wsj.com/articles/health-authorities-roll-out-new...