7 ms·

This seems like an important result, but one that we might need to approach with measured confidence. Of 14 samples, from 124 patients in Dec/Jan, one tested p

This seems like an important result, but one that we might need to approach with measured confidence. Of 14 samples, from 124 patients in Dec/Jan, one tested positive for COV by PCR. How well can we bound our uncertainty about false positive in such circumstances?

ImaCake 6y ago
I don't think we have any statistical tests that will give us any kind of certainty with a 1/14 result. At best you could assume it fits the expected value (maximum likelihood) of a binomial distribution[0], which gives us a 35% chance that the next sample of 14 such people would have no cases at all. 0. https://www.wolframalpha.com/input/?i=binomial+distribution+n%3D14+p+%3D1%2F14 https://www.wolframalpha.com/input/?i=binomial+distribution+...
- contravariant 6y ago
  You're not quite thinking in the right direction. No matter the statistical model a 1/14 result, is absolute evidence against there being no infections at all. The problem lies with the test itself, which might have an unknown false positive rate. Although in this case we're basically looking at what I understand to be the gold standard in RNA/DNA evidence, combined with matching symptoms. Also apparently they had 2 separate teams testing the samples using different methodology, so we've got at least a decent amount of confidence that something is going on with that sample, although it doesn't rule out systematic bias.
  - icegreentea2 6y ago
    Yes. It's possible (I can't put a number on how likely) that the sample was mishandled and contamination introduced. Depending at what stage this occurred, this could affect all future tests of the sample. Ideally they are able to follow up with serological tests against the specific patient. While a positive carries no information (might have gotten COVID in the intervening time), a negative would point to some sort of sample contamination. This isn't a knock against their skill or their competence. Sometimes shit just happens. Certainly if this is true, I'd be very interested in how this changes the nature of our models of the virus.
    - skwb 6y ago
      Yeah, but for IRB reasons, you may not be able to contact the patient. IMHO I would prefer this (potentially) imperfect short communication being out in the scientific world now than a more perfect communication requiring several more months.
      - SiempreViernes 6y ago
        If this turns out to be a false positive from contamination, how many man-month do you estimate will be wasted on making models that don't describe reality before the error is caught?
    - Jabbles 6y ago
      If it is possible to sequence the sample they would be able to confirm that it came from a very early branch of the genetic tree, rather than one introduced at a much later stage.
Johnjonjoan 6y ago
Dr Yves Cohen stated the sample was tested twice to make sure. Can't find the exact quote I read earlier but this link says basically the same. https://www.heart.co.uk/news/coronavirus/french-covid-case-december/ https://www.heart.co.uk/news/coronavirus/french-covid-case-d...
- jonny_eh 6y ago
  What if the sample is contaminated?
  - skwb 6y ago
    Certainly a possibility, but I would think that it would very unlikely to contaminate the source sample (usually they're frozen in -80 fridge), and if you have a contaminated work station, then the rest of the samples would likely be return false-positive as well.
  - dagnabbit 6y ago
    You should read the paper. It contains an excellent answer!
ttul 6y ago
The paper provides strong evidence that the test is not a false positive. They show an amplification plot where you can easily see the positive sample and negative sample along with the patient sample. The patient sample starts ramping up after cycle 30 of the PCR amplification process. They support the COVID-19 diagnosis by looking at the patient's lung CT scan, which "revealed bilateral ground glass opacity in inferior lobes." A decent explanation of PCR amplification can be found here: https://www.promega.ca/resources/guides/nucleic-acid-analysis/pcr-amplification/ https://www.promega.ca/resources/guides/nucleic-acid-analysi... "Each cycle of PCR includes steps for template denaturation, primer annealing and primer extension. The initial step denatures the target DNA by heating it to 94°C or higher for 15 seconds to 2 minutes. In the denaturation process, the two intertwined strands of DNA separate from one another, producing the necessary single-stranded DNA template for replication by the thermostable DNA polymerase. In the next step of a cycle, the temperature is reduced to approximately 40–60°C. At this temperature, the oligonucleotide primers can form stable associations (anneal) with the denatured target DNA and serve as primers for the DNA polymerase. This step lasts approximately 15–60 seconds. Finally, the synthesis of new DNA begins as the reaction temperature is raised to the optimum for the DNA polymerase. For most thermostable DNA polymerases, this temperature is in the range of 70–74°C. The extension step lasts approximately 1–2 minutes. The next cycle begins with a return to 94°C for denaturation."

ImaCake 6y ago

I don't think we have any statistical tests that will give us any kind of certainty with a 1/14 result. At best you could assume it fits the expected value (maximum likelihood) of a binomial distribution[0], which gives us a 35% chance that the next sample of 14 such people would have no cases at all. 0. https://www.wolframalpha.com/input/?i=binomial+distribution+n%3D14+p+%3D1%2F14 https://www.wolframalpha.com/input/?i=binomial+distribution+...

contravariant 6y ago
You're not quite thinking in the right direction. No matter the statistical model a 1/14 result, is absolute evidence against there being no infections at all. The problem lies with the test itself, which might have an unknown false positive rate. Although in this case we're basically looking at what I understand to be the gold standard in RNA/DNA evidence, combined with matching symptoms. Also apparently they had 2 separate teams testing the samples using different methodology, so we've got at least a decent amount of confidence that something is going on with that sample, although it doesn't rule out systematic bias.
- icegreentea2 6y ago
  Yes. It's possible (I can't put a number on how likely) that the sample was mishandled and contamination introduced. Depending at what stage this occurred, this could affect all future tests of the sample. Ideally they are able to follow up with serological tests against the specific patient. While a positive carries no information (might have gotten COVID in the intervening time), a negative would point to some sort of sample contamination. This isn't a knock against their skill or their competence. Sometimes shit just happens. Certainly if this is true, I'd be very interested in how this changes the nature of our models of the virus.
  - skwb 6y ago
    Yeah, but for IRB reasons, you may not be able to contact the patient. IMHO I would prefer this (potentially) imperfect short communication being out in the scientific world now than a more perfect communication requiring several more months.
    - SiempreViernes 6y ago
      If this turns out to be a false positive from contamination, how many man-month do you estimate will be wasted on making models that don't describe reality before the error is caught?
  - Jabbles 6y ago
    If it is possible to sequence the sample they would be able to confirm that it came from a very early branch of the genetic tree, rather than one introduced at a much later stage.

Johnjonjoan 6y ago

Dr Yves Cohen stated the sample was tested twice to make sure. Can't find the exact quote I read earlier but this link says basically the same. https://www.heart.co.uk/news/coronavirus/french-covid-case-december/ https://www.heart.co.uk/news/coronavirus/french-covid-case-d...

jonny_eh 6y ago
What if the sample is contaminated?
- skwb 6y ago
  Certainly a possibility, but I would think that it would very unlikely to contaminate the source sample (usually they're frozen in -80 fridge), and if you have a contaminated work station, then the rest of the samples would likely be return false-positive as well.
- dagnabbit 6y ago
  You should read the paper. It contains an excellent answer!

ttul 6y ago

The paper provides strong evidence that the test is not a false positive. They show an amplification plot where you can easily see the positive sample and negative sample along with the patient sample. The patient sample starts ramping up after cycle 30 of the PCR amplification process. They support the COVID-19 diagnosis by looking at the patient's lung CT scan, which "revealed bilateral ground glass opacity in inferior lobes." A decent explanation of PCR amplification can be found here: https://www.promega.ca/resources/guides/nucleic-acid-analysis/pcr-amplification/ https://www.promega.ca/resources/guides/nucleic-acid-analysi... "Each cycle of PCR includes steps for template denaturation, primer annealing and primer extension. The initial step denatures the target DNA by heating it to 94°C or higher for 15 seconds to 2 minutes. In the denaturation process, the two intertwined strands of DNA separate from one another, producing the necessary single-stranded DNA template for replication by the thermostable DNA polymerase. In the next step of a cycle, the temperature is reduced to approximately 40–60°C. At this temperature, the oligonucleotide primers can form stable associations (anneal) with the denatured target DNA and serve as primers for the DNA polymerase. This step lasts approximately 15–60 seconds. Finally, the synthesis of new DNA begins as the reaction temperature is raised to the optimum for the DNA polymerase. For most thermostable DNA polymerases, this temperature is in the range of 70–74°C. The extension step lasts approximately 1–2 minutes. The next cycle begins with a return to 94°C for denaturation."