4 ms·

This is one single frozen sample from March 2019 that was positive. Note that subsequent frozen samples from later in 2019 were negative. Without more data -

This is one single frozen sample from March 2019 that was positive. Note that subsequent frozen samples from later in 2019 were negative. Without more data - I'd chalk this up to a false positive or sample contamination.

cjbprime 6y ago
Also seems like this could end up being a case of multiple hypotheses -- if the false positive rate is very low but all the large cities are performing this test right now, someone's going to hit what will seem to them to be a very unlucky false positive.
moralestapia 6y ago
Hard to define the null hypothesis on this one, to be honest ...
barbegal 6y ago
I'd have to agree given the prior probability of finding a positive result only in the March 2019 sample. Extraordinary results need extraordinary proof.
hedora 6y ago
The study tested 36 samples, from what I can tell (N is well-obfuscated). About ten of those were from before the COVID outbreak. The paper doesn’t talk about the false positive rate of the test they used, and doesn’t compute any sort of P value. ~ 3% is a reasonable false positive rate for a COVID test, so you’d expect 0.3 of those earlier samples to test positive. I doubt it will survive peer review in its current form. In fairness to the authors, the focus of the paper is testing sewage to track diseases. The most interesting result is that they stopped detecting COVID mid-pandemic due to heavy rainfall. Edit: plugging into binomial distribution calculator says there’s over a 25% chance of getting one false positive in the pre-covid samples.
- pfortuny 6y ago
  (This is not tinfoil-hatting but mere elementary statistics). As long as you keep testing, you will get a false positive (the farther away you test, the more striking it gets). So, it is: if you keep testing, you will find something. It might as well have been "There were SARS-Covid-2 positive samples in June 2018", but it was March 2019.
- jcahill 6y ago
  They used five assays. All of these were from the WHO in-house assays list. In theory, their primers shouldn't have specificity issues. In practice, it's more complicated than that. The CDC 2019-nCoV rRT-PCR assays have a particularly sketchy history. The WHO in-house assays compendium[4] explicitly states that no false positives are expected for these. But there are huge, annoying caveats to this. (Tedium warning) Caveat A: Information chaos. This information has become substantially harder to find on CDC's end than it was originally, having been reorganized, replaced, and removed repeatedly at this point.[1] We can expect this confusion to have forced downstream errors on the part of at least some labs attempting to use the protocols. Caveat B: The CDC's initial primers[3] were widely criticized for having been badly selected. Example.[5] Caveat C: The CDC's primers have changed. Without the preprint authors providing more specific information, it's impossible to say what they were testing against for this reason. The preprint authors cite the fda.gov replacement pdf[6], which in turn links out to the CDC primers and probes page[3] via [7] > More Resources on the CDC Diagnostic Panel > Research Use Only RT-PCR Primers and Probes. So all we know is that they used CDC primers, but not which revision. [1]: CDC replaced its original instructions file, which was located at[2], with a "redirect" pdf containing a link to a replacement fda.gov page. You may find the original document by searching "Tobacco Specific Nitrosamines in Smoke". The reason for this is that the title metadata of CDC's PDFs is often wrong, since they borrow from other files while compiling them. [2]: URL: https://cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel-for-detection-instructions.pdf https://cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel... Contents: Real-Time RT-PCR Panel for Detection 2019-Novel Coronavirus - Instructions for Use PDF Title: Tobacco Specific Nitrosamines in Smoke [3]: URL: https://cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel-primer-probes.pdf https://cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel... Contents: 2019-Novel Coronavirus (2019-nCoV) Real-time rRT-PCR Panel Primers and Probes PDF Title: Avian H7 (Eurasian Lineage) Influenza Real-time RT-PCR Panel Primer Probes [4]: The compendium document that compiles protocols is a PDF. URL (landing page): https://www.who.int/publications/m/item/molecular-assays-to-diagnose-covid-19-summary-table-of-available-protocols https://www.who.int/publications/m/item/molecular-assays-to-... Section: "2019-nCoV rRT-PCR Diagnostic Panel Results Interpretation Guide" [5]: https://tomeraltman.net/2020/03/03/technical-problems-COVID-primers.html https://tomeraltman.net/2020/03/03/technical-problems-COVID-... [6]: https://www.fda.gov/media/134922/download https://www.fda.gov/media/134922/download [7]: https://www.cdc.gov/coronavirus/2019-nCoV/lab/virus-requests.html https://www.cdc.gov/coronavirus/2019-nCoV/lab/virus-requests...
- hcknwscommenter 6y ago
  "~ 3% is a reasonable false positive rate for a COVID test" There is no way they are getting a 3% false positive rate. That's way too high for a 5 product RTPCR test. Not even the serology tests are that bad.
jcahill 6y ago
Mislabeling is also an option.

cjbprime 6y ago

Also seems like this could end up being a case of multiple hypotheses -- if the false positive rate is very low but all the large cities are performing this test right now, someone's going to hit what will seem to them to be a very unlucky false positive.

moralestapia 6y ago

Hard to define the null hypothesis on this one, to be honest ...

barbegal 6y ago

I'd have to agree given the prior probability of finding a positive result only in the March 2019 sample. Extraordinary results need extraordinary proof.

hedora 6y ago

The study tested 36 samples, from what I can tell (N is well-obfuscated). About ten of those were from before the COVID outbreak. The paper doesn’t talk about the false positive rate of the test they used, and doesn’t compute any sort of P value. ~ 3% is a reasonable false positive rate for a COVID test, so you’d expect 0.3 of those earlier samples to test positive. I doubt it will survive peer review in its current form. In fairness to the authors, the focus of the paper is testing sewage to track diseases. The most interesting result is that they stopped detecting COVID mid-pandemic due to heavy rainfall. Edit: plugging into binomial distribution calculator says there’s over a 25% chance of getting one false positive in the pre-covid samples.

pfortuny 6y ago
(This is not tinfoil-hatting but mere elementary statistics). As long as you keep testing, you will get a false positive (the farther away you test, the more striking it gets). So, it is: if you keep testing, you will find something. It might as well have been "There were SARS-Covid-2 positive samples in June 2018", but it was March 2019.
jcahill 6y ago
They used five assays. All of these were from the WHO in-house assays list. In theory, their primers shouldn't have specificity issues. In practice, it's more complicated than that. The CDC 2019-nCoV rRT-PCR assays have a particularly sketchy history. The WHO in-house assays compendium[4] explicitly states that no false positives are expected for these. But there are huge, annoying caveats to this. (Tedium warning) Caveat A: Information chaos. This information has become substantially harder to find on CDC's end than it was originally, having been reorganized, replaced, and removed repeatedly at this point.[1] We can expect this confusion to have forced downstream errors on the part of at least some labs attempting to use the protocols. Caveat B: The CDC's initial primers[3] were widely criticized for having been badly selected. Example.[5] Caveat C: The CDC's primers have changed. Without the preprint authors providing more specific information, it's impossible to say what they were testing against for this reason. The preprint authors cite the fda.gov replacement pdf[6], which in turn links out to the CDC primers and probes page[3] via [7] > More Resources on the CDC Diagnostic Panel > Research Use Only RT-PCR Primers and Probes. So all we know is that they used CDC primers, but not which revision. [1]: CDC replaced its original instructions file, which was located at[2], with a "redirect" pdf containing a link to a replacement fda.gov page. You may find the original document by searching "Tobacco Specific Nitrosamines in Smoke". The reason for this is that the title metadata of CDC's PDFs is often wrong, since they borrow from other files while compiling them. [2]: URL: https://cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel-for-detection-instructions.pdf https://cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel... Contents: Real-Time RT-PCR Panel for Detection 2019-Novel Coronavirus - Instructions for Use PDF Title: Tobacco Specific Nitrosamines in Smoke [3]: URL: https://cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel-primer-probes.pdf https://cdc.gov/coronavirus/2019-ncov/downloads/rt-pcr-panel... Contents: 2019-Novel Coronavirus (2019-nCoV) Real-time rRT-PCR Panel Primers and Probes PDF Title: Avian H7 (Eurasian Lineage) Influenza Real-time RT-PCR Panel Primer Probes [4]: The compendium document that compiles protocols is a PDF. URL (landing page): https://www.who.int/publications/m/item/molecular-assays-to-diagnose-covid-19-summary-table-of-available-protocols https://www.who.int/publications/m/item/molecular-assays-to-... Section: "2019-nCoV rRT-PCR Diagnostic Panel Results Interpretation Guide" [5]: https://tomeraltman.net/2020/03/03/technical-problems-COVID-primers.html https://tomeraltman.net/2020/03/03/technical-problems-COVID-... [6]: https://www.fda.gov/media/134922/download https://www.fda.gov/media/134922/download [7]: https://www.cdc.gov/coronavirus/2019-nCoV/lab/virus-requests.html https://www.cdc.gov/coronavirus/2019-nCoV/lab/virus-requests...
hcknwscommenter 6y ago
"~ 3% is a reasonable false positive rate for a COVID test" There is no way they are getting a 3% false positive rate. That's way too high for a 5 product RTPCR test. Not even the serology tests are that bad.

jcahill 6y ago

Mislabeling is also an option.