5 ms·

Because you quoted the article using BioMedomics' antibody diagnostic test, I will quote the accuracy from their site[0]: > In order to test the detection sens

Because you quoted the article using BioMedomics' antibody diagnostic test, I will quote the accuracy from their site[0]: > In order to test the detection sensitivity and specificity of the COVID-19 IgG-IgM combined antibody test, blood samples were collected from COVID-19 patients from multiple hospitals and Chinese CDC laboratories. The tests were done separately at each site. A total of 525 cases were tested: 397 (positive) clinically confirmed (including PCR test) SARS-CoV-2-infected patients and 128 non- SARS-CoV-2-infected patients (128 negative). The testing results of vein blood without viral inactivation were summarized in the Table 1. Of the 397 blood sample from SARS-CoV-2-infected patients, 352 tested positive, resulting in a sensitivity of 88.66%. Twelve of the blood samples from the 128 non-SARS-CoV-2 infection patients tested positive, generating a specificity of 90.63%. [0]: https://www.biomedomics.com/products/infectious-disease/covid-19-rt/ https://www.biomedomics.com/products/infectious-disease/covi...

deleted 6y ago
[deleted]
skybrian 6y ago
Specificity of 90% seems really low? Does that mean almost 1 in 10 would test positive even if nobody had it?
- DenisM 6y ago
  Yes, it's 10% false positive. But it's not "really low" in the sense that it's typical for antibody tests. Which is one reason not to do wide-spread testing.
  - subroutine 6y ago
    Or you could flip that around and make testing so plentiful everyone is tested 3x (perhaps by independent labs), so the FP rate is more like... 1/10 * 1/10 * 1/10 (1/1000)
    - notforever 6y ago
      No good reason for test errors to be uncorrelated, ie. quite possible same people who trigger false positives on the first test would be false positives on the second etc.
      - subroutine 6y ago
        Sure, there will probably be some correlation. Depends heavily on how the 3x testing is designed. T1. Drive up saliva swab by nurse, for serology test. T2. Re-deploy census peeps to get samples (and census info) from entire neighborhoods. T3. 23andMe-like home kit express mail to lab The odds of people getting a FP on all three might not be 1 in 1000, but it probably isn't 1 in 10. Anyway, as it is, leading 1 in 10 people to falsely believe they are immune, isn't much better than having no test at all.
    - zucker42 6y ago
      Assuming false positives aren't correlated even with independently manufactured tests seems like a terrible idea to me, knowing little about medicine.
      - subroutine 6y ago
        Sounds like you had a bad experience once, with 'double checking'.
    - gmueckl 6y ago
      This assumes that the cause for false positive is in the test execution and not a contamination in the sample itself.
    - ejstronge 6y ago
      This would require that false positives are truly random and not influenced by sample-specific factors. Is this the case? In antibody testing, I would imagine that false positives result from cross-reactivity instead of random chance.
- feral 6y ago
  That's a correct understanding of specificity. That would imply the correct rate is closer to 22% than 30%, as of the 64 positives out of 200, we would expect roughly 20 of those to be false positives. (So the remaining 44/200=>22%). Article says there are 40k residents and 39 deaths. If 22% of 40k are infected => 8800 infected. 37/8800 implies .42% IFR, broadly consistent with figures elsewhere, especially given that there is a lag time for deaths. (Which will cause the IFR to increase.) Any bias due to selection etc would basically decrease the denominator on that calculation, increasing the IFR. (This is all just back of napkin, based on numbers in the article, please check my calculations.)
michaeloder 6y ago
A 10% false-positive rate is awful. If the actual incidence is 2%, a test with that error rate could overstate the results at about 12%. That combined with the sampling bias could easily result in overstated numbers. Randomized household sampling would be far preferable. That would obviously take much more time and would expose testers and the household to more risk. But without good methods, research like this and the surveys conducted in Santa Clara and LA counties are potentially worst than useless since they have the potential of misinforming policymakers and the public.
sct202 6y ago
So interestingly, those results from the research paper with more details ( https://www.biomedomics.com/?fldl=3050 https://www.biomedomics.com/?fldl=3050 and also https://onlinelibrary.wiley.com/doi/full/10.1002/jmv.25727 https://onlinelibrary.wiley.com/doi/full/10.1002/jmv.25727 ) credit a different manufacturer called Jiangsu Medomics Medical Technologies. I wonder if that's their subsidiary, or if they're buying the tests from them.
jariel 6y ago
This specificity issue is bothering me. This 'medical' meaning 'very important' - so if there's a test with 90% accuracy, why on heaven's earth do we not simply run the test 3 times to get 'considerably greater accuracy'? 10% error is so large it's tough to make heads or tails of the data? Does someone know if this makes sense i.e. if we can simply run the test 3 times and get better data. Secondly - why are these health authorities not doing proper, state-wide tests? Here we are with an economy in meltdown, trying to 'model model model' with a 10 Trillion dollar economy wouldn't it make sense for at least ONE (or a few) freaking comprehensive tests that give us some good data?

deleted 6y ago

[deleted]

skybrian 6y ago

Specificity of 90% seems really low? Does that mean almost 1 in 10 would test positive even if nobody had it?

DenisM 6y ago
Yes, it's 10% false positive. But it's not "really low" in the sense that it's typical for antibody tests. Which is one reason not to do wide-spread testing.
- subroutine 6y ago
  Or you could flip that around and make testing so plentiful everyone is tested 3x (perhaps by independent labs), so the FP rate is more like... 1/10 * 1/10 * 1/10 (1/1000)
  - notforever 6y ago
    No good reason for test errors to be uncorrelated, ie. quite possible same people who trigger false positives on the first test would be false positives on the second etc.
    - subroutine 6y ago
      Sure, there will probably be some correlation. Depends heavily on how the 3x testing is designed. T1. Drive up saliva swab by nurse, for serology test. T2. Re-deploy census peeps to get samples (and census info) from entire neighborhoods. T3. 23andMe-like home kit express mail to lab The odds of people getting a FP on all three might not be 1 in 1000, but it probably isn't 1 in 10. Anyway, as it is, leading 1 in 10 people to falsely believe they are immune, isn't much better than having no test at all.
  - zucker42 6y ago
    Assuming false positives aren't correlated even with independently manufactured tests seems like a terrible idea to me, knowing little about medicine.
    - subroutine 6y ago
      Sounds like you had a bad experience once, with 'double checking'.
  - gmueckl 6y ago
    This assumes that the cause for false positive is in the test execution and not a contamination in the sample itself.
  - ejstronge 6y ago
    This would require that false positives are truly random and not influenced by sample-specific factors. Is this the case? In antibody testing, I would imagine that false positives result from cross-reactivity instead of random chance.
feral 6y ago
That's a correct understanding of specificity. That would imply the correct rate is closer to 22% than 30%, as of the 64 positives out of 200, we would expect roughly 20 of those to be false positives. (So the remaining 44/200=>22%). Article says there are 40k residents and 39 deaths. If 22% of 40k are infected => 8800 infected. 37/8800 implies .42% IFR, broadly consistent with figures elsewhere, especially given that there is a lag time for deaths. (Which will cause the IFR to increase.) Any bias due to selection etc would basically decrease the denominator on that calculation, increasing the IFR. (This is all just back of napkin, based on numbers in the article, please check my calculations.)

michaeloder 6y ago

A 10% false-positive rate is awful. If the actual incidence is 2%, a test with that error rate could overstate the results at about 12%. That combined with the sampling bias could easily result in overstated numbers. Randomized household sampling would be far preferable. That would obviously take much more time and would expose testers and the household to more risk. But without good methods, research like this and the surveys conducted in Santa Clara and LA counties are potentially worst than useless since they have the potential of misinforming policymakers and the public.

sct202 6y ago

So interestingly, those results from the research paper with more details ( https://www.biomedomics.com/?fldl=3050 https://www.biomedomics.com/?fldl=3050 and also https://onlinelibrary.wiley.com/doi/full/10.1002/jmv.25727 https://onlinelibrary.wiley.com/doi/full/10.1002/jmv.25727 ) credit a different manufacturer called Jiangsu Medomics Medical Technologies. I wonder if that's their subsidiary, or if they're buying the tests from them.

jariel 6y ago

This specificity issue is bothering me. This 'medical' meaning 'very important' - so if there's a test with 90% accuracy, why on heaven's earth do we not simply run the test 3 times to get 'considerably greater accuracy'? 10% error is so large it's tough to make heads or tails of the data? Does someone know if this makes sense i.e. if we can simply run the test 3 times and get better data. Secondly - why are these health authorities not doing proper, state-wide tests? Here we are with an economy in meltdown, trying to 'model model model' with a 10 Trillion dollar economy wouldn't it make sense for at least ONE (or a few) freaking comprehensive tests that give us some good data?