4 ms·

It must have taken a special effort to write all those words without once mentioning the false positive rate of the PCR test (~2.3%), which (unlike earlier in t

by alvah 6y ago

It must have taken a special effort to write all those words without once mentioning the false positive rate of the PCR test (~2.3%), which (unlike earlier in the pandemic) is now in the ballpark of reported case numbers: https://www.medrxiv.org/content/10.1101/2020.04.26.20080911v3 https://www.medrxiv.org/content/10.1101/2020.04.26.20080911v...

dgfitz 6y ago
I don’t think I understand. Would factoring that in have significantly affected the thesis of the article?
- zamadatix 6y ago
  I think the implication is if true false positive rate closely matches the reported positive result rate there haven't actually been a growing number of cases just a growing number of tests. I'm not sure the linked material validates this claim strongly though but I also haven't put in the time to see if it doesn't either.
  - dgfitz 6y ago
    Oh ok I understand now, thanks for explaining!
- henriquez 6y ago
  I think the GP is suggesting that the actual case rate is lower than the numbers suggest due to the unknown rate of false positives.
H8crilA 6y ago
Wow, thank you for that, I had no idea PCR is this inaccurate. The process always created a very "pure" image in my head, and now I feel kind of stupid never checking how inaccurate it is in practice. Of course I'm not a professional in the area. How does it happen on a molecular level that PCR delivers a false positive? Are the primers sometimes broken (matching not exactly what they're supposed to match)?
- khuey 6y ago
  A properly performed PCR test is essentially 100% specific. False positives come from contamination of samples, mislabelled samples, and other forms of human error.
  - deleted 6y ago
    [deleted]
- mehrdadn 6y ago
  > Wow, thank you for that, I had no idea PCR is this inaccurate. I'm not sure it really is. Take it with a grain of salt for now. I feel like there's a missing piece of the puzzle here.
tuna-piano 6y ago
If the false positive of the PCR test really is 2.3%, how are so many places able to have positivity rates consistently less than that, more like .1%[1]? And how can test pooling in huge numbers (like was done in Wuhan) work? A false positive rate that high doesn't pass the sniff test. [1]https://ourworldindata.org/coronavirus-testing https://ourworldindata.org/coronavirus-testing [2]https://www.nytimes.com/2020/05/26/world/asia/coronavirus-wuhan-tests.html https://www.nytimes.com/2020/05/26/world/asia/coronavirus-wu...
- wjn0 6y ago
  I'm not sure if this is it, but assuming the correctness of the test is independent of the sample quality, a sample could be split and tested multiple times to obtain a result with a much better FPR.
  - mehrdadn 6y ago
    Anecdotally I've heard this isn't the case, but I don't have links.
  - cameldrv 6y ago
    To my knowledge, the most common failure mode is that the concentration of viral RNA in the sample is below the limit of detection. This could be because the swab wasn't done correctly, the patient wasn't shedding very much virus, the sample was mishandled (heat can break down the sample), or the sample wasn't adequately mixed.
- goalieca 6y ago
  There is a relatively massive false negative rate.
  - repsilat 6y ago
    Regardless of false negative rate, measured positives are always at least as frequent as false positives.
- nl 6y ago
pps43 6y ago
The tests themselves are fine, false positives are due to human error: "The high specificities (usually 100%) reported in PCR-based tests for SARS-CoV-2 infection do not represent the real-world use of these tests, where contamination and human error produce significant rates of false positives." New York reports positivity rate below 1% for a while now, so false positive rate cannot be 2.3%.
- cameldrv 6y ago
  If the false positive mechanism is contaminating one sample with another, the false positive rate will depend on the true positive rate in the samples that the lab/testing site is processing. The other possibility would be if positive control amplicon contaminated the lab, which does happen.
  - pps43 6y ago
    The underlying assumption behind false positive rate is that tests are independent and identically distributed. If in practice this is not the case, then false positive rate becomes misleading.
mehrdadn 6y ago
Hm, the numbers I've dug up (though rather hastily; I haven't read much of the documents) don't really match 2.3%... they seem to be 0% in their tests. Here [1] is where I think you can find the info for all the FDA-authorized tests. I see mentions of NPA (negative percent agreement) being 100% in their trials in [2] and [3] for example, though the sample sizes don't seem too large (something like 30-50). Any idea what I'm missing, or if it's just because of small sample sizes? (I'm not exactly trained in this, but my impression was PCR was more accurate than that.) [1] https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas#individual-molecular https://www.fda.gov/medical-devices/coronavirus-disease-2019... [2] https://www.fda.gov/media/141951/download https://www.fda.gov/media/141951/download [3] https://www.fda.gov/media/141760/download https://www.fda.gov/media/141760/download
- nl 6y ago
  This is really useful, since it's the first source that includes the Ct for each(?) test. Combine that with [1] which makes a good argument that some high Ct levels may be too sensitive. Just randomly looking, this test[2] approved in June uses the Ct of 40 which is criticised above. [1] https://www.advisory.com/daily-briefing/2020/09/01/covid-tests https://www.advisory.com/daily-briefing/2020/09/01/covid-tes... [2] https://www.fda.gov/media/139572/download https://www.fda.gov/media/139572/download
nl 6y ago
This isn't completely correct. The PCR test can produce high numbers of false positives. It depends on a variable called the "cycle threshold" (Ct), which is the number of times amplification is run. Different tests have different Ct numbers, so some specific tests are overly sensitive. However there is no data on what test is used where, or when tests were changed. https://www.advisory.com/daily-briefing/2020/09/01/covid-tests https://www.advisory.com/daily-briefing/2020/09/01/covid-tes...
epistasis 6y ago
I've seen lots of clinics testing numbers, but I've never, literally never seen a PCR test with a FP rate that high. It kind of beggars belief. This article doesn't even deal with SARS-CoV-2, so until it gets peer reviewed and, more importantly, replicated, it should be treated as an extraordinary claim that requires extraordinary evidence.
rsynnott 6y ago
I doubt that’s it. Ireland’s positive rate was about 0.2% for a few weeks, now back up to 1.6%.

by alvah 6y ago

dgfitz 6y ago

I don’t think I understand. Would factoring that in have significantly affected the thesis of the article?

zamadatix 6y ago
I think the implication is if true false positive rate closely matches the reported positive result rate there haven't actually been a growing number of cases just a growing number of tests. I'm not sure the linked material validates this claim strongly though but I also haven't put in the time to see if it doesn't either.
- dgfitz 6y ago
  Oh ok I understand now, thanks for explaining!
henriquez 6y ago
I think the GP is suggesting that the actual case rate is lower than the numbers suggest due to the unknown rate of false positives.

H8crilA 6y ago

Wow, thank you for that, I had no idea PCR is this inaccurate. The process always created a very "pure" image in my head, and now I feel kind of stupid never checking how inaccurate it is in practice. Of course I'm not a professional in the area. How does it happen on a molecular level that PCR delivers a false positive? Are the primers sometimes broken (matching not exactly what they're supposed to match)?

khuey 6y ago
A properly performed PCR test is essentially 100% specific. False positives come from contamination of samples, mislabelled samples, and other forms of human error.
- deleted 6y ago
  [deleted]
mehrdadn 6y ago
> Wow, thank you for that, I had no idea PCR is this inaccurate. I'm not sure it really is. Take it with a grain of salt for now. I feel like there's a missing piece of the puzzle here.

tuna-piano 6y ago

If the false positive of the PCR test really is 2.3%, how are so many places able to have positivity rates consistently less than that, more like .1%[1]? And how can test pooling in huge numbers (like was done in Wuhan) work? A false positive rate that high doesn't pass the sniff test. [1]https://ourworldindata.org/coronavirus-testing https://ourworldindata.org/coronavirus-testing [2]https://www.nytimes.com/2020/05/26/world/asia/coronavirus-wuhan-tests.html https://www.nytimes.com/2020/05/26/world/asia/coronavirus-wu...

wjn0 6y ago
I'm not sure if this is it, but assuming the correctness of the test is independent of the sample quality, a sample could be split and tested multiple times to obtain a result with a much better FPR.
- mehrdadn 6y ago
  Anecdotally I've heard this isn't the case, but I don't have links.
- cameldrv 6y ago
  To my knowledge, the most common failure mode is that the concentration of viral RNA in the sample is below the limit of detection. This could be because the swab wasn't done correctly, the patient wasn't shedding very much virus, the sample was mishandled (heat can break down the sample), or the sample wasn't adequately mixed.
goalieca 6y ago
There is a relatively massive false negative rate.
- repsilat 6y ago
  Regardless of false negative rate, measured positives are always at least as frequent as false positives.
nl 6y ago

pps43 6y ago

The tests themselves are fine, false positives are due to human error: "The high specificities (usually 100%) reported in PCR-based tests for SARS-CoV-2 infection do not represent the real-world use of these tests, where contamination and human error produce significant rates of false positives." New York reports positivity rate below 1% for a while now, so false positive rate cannot be 2.3%.

cameldrv 6y ago
If the false positive mechanism is contaminating one sample with another, the false positive rate will depend on the true positive rate in the samples that the lab/testing site is processing. The other possibility would be if positive control amplicon contaminated the lab, which does happen.
- pps43 6y ago
  The underlying assumption behind false positive rate is that tests are independent and identically distributed. If in practice this is not the case, then false positive rate becomes misleading.

mehrdadn 6y ago

Hm, the numbers I've dug up (though rather hastily; I haven't read much of the documents) don't really match 2.3%... they seem to be 0% in their tests. Here [1] is where I think you can find the info for all the FDA-authorized tests. I see mentions of NPA (negative percent agreement) being 100% in their trials in [2] and [3] for example, though the sample sizes don't seem too large (something like 30-50). Any idea what I'm missing, or if it's just because of small sample sizes? (I'm not exactly trained in this, but my impression was PCR was more accurate than that.) [1] https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas#individual-molecular https://www.fda.gov/medical-devices/coronavirus-disease-2019... [2] https://www.fda.gov/media/141951/download https://www.fda.gov/media/141951/download [3] https://www.fda.gov/media/141760/download https://www.fda.gov/media/141760/download

nl 6y ago
This is really useful, since it's the first source that includes the Ct for each(?) test. Combine that with [1] which makes a good argument that some high Ct levels may be too sensitive. Just randomly looking, this test[2] approved in June uses the Ct of 40 which is criticised above. [1] https://www.advisory.com/daily-briefing/2020/09/01/covid-tests https://www.advisory.com/daily-briefing/2020/09/01/covid-tes... [2] https://www.fda.gov/media/139572/download https://www.fda.gov/media/139572/download

nl 6y ago

This isn't completely correct. The PCR test can produce high numbers of false positives. It depends on a variable called the "cycle threshold" (Ct), which is the number of times amplification is run. Different tests have different Ct numbers, so some specific tests are overly sensitive. However there is no data on what test is used where, or when tests were changed. https://www.advisory.com/daily-briefing/2020/09/01/covid-tests https://www.advisory.com/daily-briefing/2020/09/01/covid-tes...

epistasis 6y ago

I've seen lots of clinics testing numbers, but I've never, literally never seen a PCR test with a FP rate that high. It kind of beggars belief. This article doesn't even deal with SARS-CoV-2, so until it gets peer reviewed and, more importantly, replicated, it should be treated as an extraordinary claim that requires extraordinary evidence.

rsynnott 6y ago

I doubt that’s it. Ireland’s positive rate was about 0.2% for a few weeks, now back up to 1.6%.