5 ms·

> We tested 1,020 serum specimens collected prior to SARS-CoV-2 circulation in the United States and found one false positive This gives 95% confidence interva

> We tested 1,020 serum specimens collected prior to SARS-CoV-2 circulation in the United States and found one false positive This gives 95% confidence intervals of between 0.0% and 0.5% false positive rate.

ghshephard 6y ago
Okay - I'm going to reveal my statistics ignorance here - but if they did get a false positive, how does 0.0% fall within the range of possibilities (or is that allowing for it might be 0.0499..% or lower?)
- samatman 6y ago
  All of the sibling answers are correct, but ignoring another possibility: that the timeline is simply wrong, and SARS2 was present in the US early enough that the "false positive" is in fact a true positive. Which can't lead to a "true 0%", since there's no such animal. But it could mean that all of the tests gave correct answers, in the trial.
  - squnch 6y ago
    They could be testing this on five year old samples.
    - MarkusQ 6y ago
      From the article: > Specificity samples were derived from de-identified excess serum specimens sent to our clinical virology laboratory in 2018 and 2019. So...not five years ago, and it is even possible that a significant number were from late 2019.
- richardwhiuk 6y ago
  Rounding - 0.000000001% is treated as 0.0%. (More likely, 0.04% or similar is being treated as 0.0%, but I haven't done the maths to confirm what the number would be)
- IfOnlyYouKnew 6y ago
  Not ignorant at all... I believe the issue is that the statement ("95% confidence interval of 0.0 to 0.5") has just slightly different/counterintuitive meaning in this case. In general, you can always make the interval larger, but you want it to be smaller. With the understanding that the 95% is a lower bound, [0.0, 25%] is also a valid confidence interval. But, yes, you are entirely correct as to the practical meaning. 0% itself is impossible, but 0.0...1% could still happen.
- renewiltord 6y ago
  What is the smallest number greater than 0? For the things considered here, the point 0 is measure zero so open starting at 0 and closed at 0 are the same. L
- barbegal 6y ago
  Correct, I rounded to the appropriate number of significant figures. I hate extra significant figures which suggest that the data is more precise than it really is
- madhadron 6y ago
  We know that 0% isn't a real value for a test's error rate, but the statistical procedure being used doesn't have any "look, it's never zero" criteria built into its math. All it says is that 0 is one of the numbers consistent with the data given these criteria.
cozzyd 6y ago
Now is a perfect time for people to bone up on the rabbit hole that is binomial confidence intervals! https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval https://en.wikipedia.org/wiki/Binomial_proportion_confidence... TLDR: whether you're a frequentist or Bayesian, the Jeffrey's interval is pretty good. Use Clopper-Pearson if you're a frequentist who is scared of undercovering. I get 0.011% for the 2.5% quantile and 0.46% for the 97.5th quantile.

ghshephard 6y ago

Okay - I'm going to reveal my statistics ignorance here - but if they did get a false positive, how does 0.0% fall within the range of possibilities (or is that allowing for it might be 0.0499..% or lower?)

samatman 6y ago
All of the sibling answers are correct, but ignoring another possibility: that the timeline is simply wrong, and SARS2 was present in the US early enough that the "false positive" is in fact a true positive. Which can't lead to a "true 0%", since there's no such animal. But it could mean that all of the tests gave correct answers, in the trial.
- squnch 6y ago
  They could be testing this on five year old samples.
  - MarkusQ 6y ago
    From the article: > Specificity samples were derived from de-identified excess serum specimens sent to our clinical virology laboratory in 2018 and 2019. So...not five years ago, and it is even possible that a significant number were from late 2019.
richardwhiuk 6y ago
Rounding - 0.000000001% is treated as 0.0%. (More likely, 0.04% or similar is being treated as 0.0%, but I haven't done the maths to confirm what the number would be)
IfOnlyYouKnew 6y ago
Not ignorant at all... I believe the issue is that the statement ("95% confidence interval of 0.0 to 0.5") has just slightly different/counterintuitive meaning in this case. In general, you can always make the interval larger, but you want it to be smaller. With the understanding that the 95% is a lower bound, [0.0, 25%] is also a valid confidence interval. But, yes, you are entirely correct as to the practical meaning. 0% itself is impossible, but 0.0...1% could still happen.
renewiltord 6y ago
What is the smallest number greater than 0? For the things considered here, the point 0 is measure zero so open starting at 0 and closed at 0 are the same. L
barbegal 6y ago
Correct, I rounded to the appropriate number of significant figures. I hate extra significant figures which suggest that the data is more precise than it really is
madhadron 6y ago
We know that 0% isn't a real value for a test's error rate, but the statistical procedure being used doesn't have any "look, it's never zero" criteria built into its math. All it says is that 0 is one of the numbers consistent with the data given these criteria.

cozzyd 6y ago

Now is a perfect time for people to bone up on the rabbit hole that is binomial confidence intervals! https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval https://en.wikipedia.org/wiki/Binomial_proportion_confidence... TLDR: whether you're a frequentist or Bayesian, the Jeffrey's interval is pretty good. Use Clopper-Pearson if you're a frequentist who is scared of undercovering. I get 0.011% for the 2.5% quantile and 0.46% for the 97.5th quantile.