3 ms·

I made this comment the last time a serological testing study came out: they have 30 negative controls and claim 1.5% positives among their tests. So the number

by tgb 7y ago

I made this comment the last time a serological testing study came out: they have 30 negative controls and claim 1.5% positives among their tests. So the number of controls is insufficient to rule out this being entirely false positives. You have to then rely on the test manufacturer's claims of false positives rates, which comes from 371 negative controls. They say: > our estimates of specificity are 99.5% (95 CI 98.1-99.9%) and 100% (95 CI 90.5-100%) Look at the those confidence intervals, even the narrower one (from the manufacturer's data)! The bottom is a four-fold increase in false positives, compared to the point-estimate, and is greater than the total positives they had in their finding.

tom-thistime 7y ago
This (I think?). I am not a doctor or biologist. Anyone who knows more than me, which is not a high bar, please step in and correct me. Based on quick amateur reading of the paper, the rate of crude positives (1.5%) might be entirely real cases, entirely false positives, or anywhere in between. The real rate of infection could be anywhere from 0% to 5%. The authors report around 3,500 real tests and 30 known-negative tests. We need 30,000 real data points (or 300 known-negative tests) to conclude anything. (Right?) I am not criticizing the experts who conducted the study. Thank you for doing the study. I'm only trying to understand what the study reveals. It may reveal that the group needs 10x more funding right away.
ajross 7y ago
To repeat you in plainer English: the serum test isn't infallible and reports "infected", incorrectly, for uninfected patients. It does this at some rate that is very near, or perhaps larger than, the fraction of "true" infected patients they report in their results. Teasing out data from all that noise requires that the false positive error rate be measured very accurately. And they didn't do that, so really this doesn't tell us much. Edit: Alternatively, borrowing jargon from a more common field around here: the measured infection rate of ~3% is very close to the noise floor of the experiment. It might be that, or it might be near zero, and we can't tell the difference. This study is very good evidence that the infection fraction is not much larger, however. We can easily rule out high infection rates like the 30% numbers that seems to get thrown around.

by tgb 7y ago

tom-thistime 7y ago

This (I think?). I am not a doctor or biologist. Anyone who knows more than me, which is not a high bar, please step in and correct me. Based on quick amateur reading of the paper, the rate of crude positives (1.5%) might be entirely real cases, entirely false positives, or anywhere in between. The real rate of infection could be anywhere from 0% to 5%. The authors report around 3,500 real tests and 30 known-negative tests. We need 30,000 real data points (or 300 known-negative tests) to conclude anything. (Right?) I am not criticizing the experts who conducted the study. Thank you for doing the study. I'm only trying to understand what the study reveals. It may reveal that the group needs 10x more funding right away.

ajross 7y ago

To repeat you in plainer English: the serum test isn't infallible and reports "infected", incorrectly, for uninfected patients. It does this at some rate that is very near, or perhaps larger than, the fraction of "true" infected patients they report in their results. Teasing out data from all that noise requires that the false positive error rate be measured very accurately. And they didn't do that, so really this doesn't tell us much. Edit: Alternatively, borrowing jargon from a more common field around here: the measured infection rate of ~3% is very close to the noise floor of the experiment. It might be that, or it might be near zero, and we can't tell the difference. This study is very good evidence that the infection fraction is not much larger, however. We can easily rule out high infection rates like the 30% numbers that seems to get thrown around.