8 ms·

Anecdotes aren't worth much in this area. There's no conspiracy of silence about problems. We can use actual science. They actually test the quality of these te

Anecdotes aren't worth much in this area. There's no conspiracy of silence about problems. We can use actual science. They actually test the quality of these tests. Some are better than others. For example, the Abbott test has a surprising accuracy [1] "Researchers at the University of Washington School of Medicine found Abbott’s test had a specificity rate of 99.9% and a sensitivity rate of 100%, suggesting very few chances of incorrectly diagnosing a healthy person with the infection and no false negatives." 1. https://www.cnbc.com/2020/05/08/study-suggests-abbott-covid-19-antibody-test-highly-likely-to-give-correct-results.html https://www.cnbc.com/2020/05/08/study-suggests-abbott-covid-...

9nGQluzmnq3M 6y ago
COVID test accuracy is very similar to condom effectiveness: they're both high in theory, but surprisingly low in practice, not because of the science but user error. I've seen estimates as high as 10-30% for false negatives, largely due to poor swabbing practices: https://www.healthline.com/health-news/false-negatives-covid19-tests-symptoms-assume-you-have-illness https://www.healthline.com/health-news/false-negatives-covid...
- jccooper 6y ago
  That's not the sort of test under discussion here. The Abbot test and other antibody tests are blood based. I'm sure there are things in the pipeline that can go wrong, but they're much less susceptible to error. What is important is that antibodies are a sufficient but not necessary indication of past infection. There seem to be plenty of people who don't develop them when exposed, possibly due to effective T-cell immunity.
- Rebelgecko 6y ago
  FWIW I think most if not all of the antibody tests are done on blood
- disabled 6y ago
  The Abbott test mentioned above is blood drawn from serum (not a fingerprick blood test!). This is not the nasal swab test that sometimes misses positive people, as you mentioned. To avoid confusion over this issue: There is an Abbott nasal swab test, that gives results 15 minutes after being swabbed, and without "special equipment". That specific Abbott test (the instant nasal swab) has experienced dubious results in practice, even compared to more standard nasal swab tests.
- hsitz 6y ago
  First, if you're talking about swabbing, then I think you're talking about the test to see whether a person is currently infected with the virus. That's what the article you linked was regarding. The antibody tests are a different animal. They test to see whether you might have been infected in the past, by looking for antibodies in your blood. So these are blood tests, and, I expect, have less chance for user error. But also, I think lots of covid antibody tests are not high accuracy even "in theory". It seems the FDA relaxed oversight on companies producing antibody tests: ". . . after being criticized for the fumbled rollout of diagnostic tests during the start of a global pandemic, the FDA swung hard in the other direction, waiving its usual requirements and letting firms rush self-validated [antibody] tests into the market."[1] So there are antibody tests from over 200 companies out there. Some of these antibody tests are quite accurate, high sensitivity and specificity. Others, not so much. [1] https://khn.org/news/hype-collides-with-science-as-fda-tries-to-rein-in-wild-west-of-covid-blood-tests/ https://khn.org/news/hype-collides-with-science-as-fda-tries...
nradov 6y ago
I'm still suspicious there's something going on that we don't understand with those tests. Someone I know was seriously ill for several weeks starting in late March with 9 out of the 11 listed symptoms for COVID-19. She eventually recovered and then tested negative on the Abbott SARS-CoV-2 IgG antibody assay. We can't definitively rule out some other virus but based on the timing and symptoms what else could it have been? It certainly didn't present like influenza.
- brianpan 6y ago
  Occam's razor says she had the flu and not Covid-19. Influenza isn't one thing and the flu can have severe symptoms. Swine flu is also still going around. My wife's coworker (large tech company in the Bay Area) was positive for swine flu around March.
  - nradov 6y ago
    Perhaps, but I've had the flu several times myself and seen many cases in others. This sure didn't look like a flu. I should add that in this particular case the patient's doctor diagnosed her with Covid-19 on a video call based on symptoms. And the patient had a current flu immunization (I know the flu shot isn't completely effective but even when it doesn't exactly match the viral strain it usually mitigates the symptoms).
  - Marsymars 6y ago
    > Occam's razor says she had the flu and not Covid-19. And, in most places, test positivity rates. For months in my area covid-19 tests were only available for people with symptoms matching covid-19 - and those people were still testing 95%+ negative for covid-19.
- jccooper 6y ago
  It's always possible it was something else, but lack of (testable) antibodies does not rule out COVID-19 infection, as they are not always developed and/or expressed after some time. Seems like that may even be common. Since T-cell immunity is rather hard to test, you may never know.
- 6y ago
tripletao 6y ago
Independent testing of the Abbott assay found lower sensitivity, 94%. The blood samples used to establish sensitivity are also from patients who got sick enough to seek medical care, who likely have higher levels of antibodies than milder cases. It's still one of the better tests, just not as perfect as their marketing numbers imply. https://academic.oup.com/clinchem/advance-article/doi/10.1093/clinchem/hvaa120/5836557 https://academic.oup.com/clinchem/advance-article/doi/10.109...
- three_seagrass 6y ago
  Look at the 95% confidence intervals on those results. I wouldn't take a study with N=256 (103 positive, 153 negative) seriously in claiming the specificity is a couple percentage points lower than the manufacturer's standard. That's simply too low of a cohort to draw that conclusion. Even the conclusion from that study you linked was actually that the test wasn't that accurate in the first 14 days of infection (which is already known for ab tests), not that test specificity is lower.
  - tripletao 6y ago
    First, the original question was about false negatives, and therefore about the sensitivity (positive rate on true positives), not the specificity (negative rate on true negatives). So perhaps you wrote specificity by mistake? By definition, the sensitivity can be calculated only from true positive cases, so the relevant N is just the positives. There were 103 positive samples from 48 patients, and the independent paper did their math using N = 48. Abbott's marketing referred to a study of 125 patients confirmed positive by PCR, linked below. So it's more, but not dramatically so. The affiliated study doesn't report a confidence interval, but the binomial 95% confidence interval is [97.1%, 100%]. That barely overlaps with the independent study's [82.8%, 98.7%]. So perhaps the affiliated researchers just got lucky and the independent ones got unlucky, or perhaps too many of the independent paper's >=14 day samples were <17 days and they should have waited longer; though depending how cynical your priors are you might also conclude the groups were different (not to allege any misconduct, just to say there's a lot of reasonable judgment in what patients you select that could push the result around by this small amount). Regardless, the sensitivity of the test is definitely not an unqualified 100%. Even disregarding the independent study completely, that would still be a somewhat misleading claim, given the small sample size in the affiliated study and that patients with mild illness probably develop lower titers than the average patient who sought medical care over the last few months. https://www.medrxiv.org/content/10.1101/2020.04.27.20082362v1 https://www.medrxiv.org/content/10.1101/2020.04.27.20082362v...

9nGQluzmnq3M 6y ago

COVID test accuracy is very similar to condom effectiveness: they're both high in theory, but surprisingly low in practice, not because of the science but user error. I've seen estimates as high as 10-30% for false negatives, largely due to poor swabbing practices: https://www.healthline.com/health-news/false-negatives-covid19-tests-symptoms-assume-you-have-illness https://www.healthline.com/health-news/false-negatives-covid...

jccooper 6y ago
That's not the sort of test under discussion here. The Abbot test and other antibody tests are blood based. I'm sure there are things in the pipeline that can go wrong, but they're much less susceptible to error. What is important is that antibodies are a sufficient but not necessary indication of past infection. There seem to be plenty of people who don't develop them when exposed, possibly due to effective T-cell immunity.
Rebelgecko 6y ago
FWIW I think most if not all of the antibody tests are done on blood
disabled 6y ago
The Abbott test mentioned above is blood drawn from serum (not a fingerprick blood test!). This is not the nasal swab test that sometimes misses positive people, as you mentioned. To avoid confusion over this issue: There is an Abbott nasal swab test, that gives results 15 minutes after being swabbed, and without "special equipment". That specific Abbott test (the instant nasal swab) has experienced dubious results in practice, even compared to more standard nasal swab tests.
hsitz 6y ago
First, if you're talking about swabbing, then I think you're talking about the test to see whether a person is currently infected with the virus. That's what the article you linked was regarding. The antibody tests are a different animal. They test to see whether you might have been infected in the past, by looking for antibodies in your blood. So these are blood tests, and, I expect, have less chance for user error. But also, I think lots of covid antibody tests are not high accuracy even "in theory". It seems the FDA relaxed oversight on companies producing antibody tests: ". . . after being criticized for the fumbled rollout of diagnostic tests during the start of a global pandemic, the FDA swung hard in the other direction, waiving its usual requirements and letting firms rush self-validated [antibody] tests into the market."[1] So there are antibody tests from over 200 companies out there. Some of these antibody tests are quite accurate, high sensitivity and specificity. Others, not so much. [1] https://khn.org/news/hype-collides-with-science-as-fda-tries-to-rein-in-wild-west-of-covid-blood-tests/ https://khn.org/news/hype-collides-with-science-as-fda-tries...

nradov 6y ago

I'm still suspicious there's something going on that we don't understand with those tests. Someone I know was seriously ill for several weeks starting in late March with 9 out of the 11 listed symptoms for COVID-19. She eventually recovered and then tested negative on the Abbott SARS-CoV-2 IgG antibody assay. We can't definitively rule out some other virus but based on the timing and symptoms what else could it have been? It certainly didn't present like influenza.

brianpan 6y ago
Occam's razor says she had the flu and not Covid-19. Influenza isn't one thing and the flu can have severe symptoms. Swine flu is also still going around. My wife's coworker (large tech company in the Bay Area) was positive for swine flu around March.
- nradov 6y ago
  Perhaps, but I've had the flu several times myself and seen many cases in others. This sure didn't look like a flu. I should add that in this particular case the patient's doctor diagnosed her with Covid-19 on a video call based on symptoms. And the patient had a current flu immunization (I know the flu shot isn't completely effective but even when it doesn't exactly match the viral strain it usually mitigates the symptoms).
- Marsymars 6y ago
  > Occam's razor says she had the flu and not Covid-19. And, in most places, test positivity rates. For months in my area covid-19 tests were only available for people with symptoms matching covid-19 - and those people were still testing 95%+ negative for covid-19.
jccooper 6y ago
It's always possible it was something else, but lack of (testable) antibodies does not rule out COVID-19 infection, as they are not always developed and/or expressed after some time. Seems like that may even be common. Since T-cell immunity is rather hard to test, you may never know.
6y ago

tripletao 6y ago

Independent testing of the Abbott assay found lower sensitivity, 94%. The blood samples used to establish sensitivity are also from patients who got sick enough to seek medical care, who likely have higher levels of antibodies than milder cases. It's still one of the better tests, just not as perfect as their marketing numbers imply. https://academic.oup.com/clinchem/advance-article/doi/10.1093/clinchem/hvaa120/5836557 https://academic.oup.com/clinchem/advance-article/doi/10.109...

three_seagrass 6y ago
Look at the 95% confidence intervals on those results. I wouldn't take a study with N=256 (103 positive, 153 negative) seriously in claiming the specificity is a couple percentage points lower than the manufacturer's standard. That's simply too low of a cohort to draw that conclusion. Even the conclusion from that study you linked was actually that the test wasn't that accurate in the first 14 days of infection (which is already known for ab tests), not that test specificity is lower.
- tripletao 6y ago
  First, the original question was about false negatives, and therefore about the sensitivity (positive rate on true positives), not the specificity (negative rate on true negatives). So perhaps you wrote specificity by mistake? By definition, the sensitivity can be calculated only from true positive cases, so the relevant N is just the positives. There were 103 positive samples from 48 patients, and the independent paper did their math using N = 48. Abbott's marketing referred to a study of 125 patients confirmed positive by PCR, linked below. So it's more, but not dramatically so. The affiliated study doesn't report a confidence interval, but the binomial 95% confidence interval is [97.1%, 100%]. That barely overlaps with the independent study's [82.8%, 98.7%]. So perhaps the affiliated researchers just got lucky and the independent ones got unlucky, or perhaps too many of the independent paper's >=14 day samples were <17 days and they should have waited longer; though depending how cynical your priors are you might also conclude the groups were different (not to allege any misconduct, just to say there's a lot of reasonable judgment in what patients you select that could push the result around by this small amount). Regardless, the sensitivity of the test is definitely not an unqualified 100%. Even disregarding the independent study completely, that would still be a somewhat misleading claim, given the small sample size in the affiliated study and that patients with mild illness probably develop lower titers than the average patient who sought medical care over the last few months. https://www.medrxiv.org/content/10.1101/2020.04.27.20082362v1 https://www.medrxiv.org/content/10.1101/2020.04.27.20082362v...