4 ms·
Here is a basic question: how are the sensitivity and specificity (false negative and false positive rates, IIRC) of these tests determined? It seems like a le
by papeda 6y ago
Here is a basic question: how are the sensitivity and specificity (false negative and false positive rates, IIRC) of these tests determined?
It seems like a learning problem with noisy labels. Does anyone know what they do in practice? Especially chaotic situations like this where there aren't reference tests? Are there reference positive and negative samples?
One concern also referenced for the Stanford study was that, as mentioned in a USC popular writeup on this study linked elsewhere on this post,
> Premier Biotech, the manufacturer of the test that USC and L.A. County are using, tested blood from COVID-19-positive patients with a 90 to 95% accuracy rate.
They must be factoring this into the 2-5% infection rate number somewhere?
- deleted 6y ago[deleted]
- makomk 6y agoReference negative samples are relatively easy, at least in theory. Any blood samples drawn before the Covid-19 outbreak started are guaranteed to be 100% negative.
- jupp0r 6y agoIn the Stanford study, they used blood samples that were taken before the epidemic to determine the true/false negative rate, and blood from patients that tested PCR positive in the past to determine true/false positive rate.
- loser777 6y ago...which has been characterized as being analyzed in a highly suspect way [1], where only point estimates of the specificity were used. [1] https://news.ycombinator.com/item?id=22924118 https://news.ycombinator.com/item?id=22924118
- longtimegoogler 6y agoThe Stanford study didn't properly include the specificity in their predictions. See here, https://statmodeling.stat.columbia.edu/2020/04/19/fatal-flaws-in-stanford-study-of-coronavirus-prevalence/ https://statmodeling.stat.columbia.edu/2020/04/19/fatal-flaw...
- NelsonMinar 6y agoThe sensitivity and specificity of these new tests is not established. It's part of why the FDA hasn't approved the tests yet. Scientists are doing their best with tests with unknown reliability, but it's a statistical mess right now.
- arrrg 6y agoOther more squishy factors are important, too: I don’t think Facebook posts for recruitment are a good idea. This could lead to more people taking part who have been sick or who had contact with someone who had the virus, over-representing that group of people in the sample.