3 ms·
Say you have a test that determines, with some non-perfect level of accuracy, if a person is or has been president of the USA. If that test is performed only o
by M_Mouse 10y ago
Say you have a test that determines, with some non-perfect level of accuracy, if a person is or has been president of the USA.
If that test is performed only on women, the false positive rate will be 100% regardless of accuracy. All positive results would be false positives.
If the test is preformed on any man leaving or entering the oval office, then the false positive rate will likely be significantly less then 100%, as there will be a president tested multiple times in that group, and when we test positive on him, it's not a false positive.
If we test the population at large, most positive results are going to be false positives, simply because we're unlikely to actually have a former president get tested. But in theory the false positive rate could be slightly less then 100% if we got lucky.
As the probability of an actual occurrence in the test group drops to zero, the rate of false positives approaches 100%, regardless of the accuracy of the testing method.
- jsprogrammer 10y agoI think we are talking about slightly different meanings of false positive rate. I think you might be talking about the rate relative to all positive trials. I think I am talking about the rate relative to all trials that don't meet the conditions of the test. So, you survey 100 women and your test tells me that 10 of them have been president, I say that your test has an observed false positive rate of 10%.
- M_Mouse 10y agoYou’re correct in regards to terminology; apparently what I was describing is more related to the terms “positive/negative predictive value.” Your definition of false positive rate is the correct one. But I believe my argument is still valid. If we test a group comprised of only former presidents, all positives must be true positives. So our false positive rate will be 0% while our female test set has been observed to have a false positive rate of 10%. This illustrates that the sample set effects the observed false positive rate. As the rate of occurrence of what we are testing for approaches 100% in the sample set, the false positive rate drops to zero. Inversely, as the incidence of what we are testing for in a (sufficiently large) sample set decreases from 100%, the false positive rate will eventually increase to some non-zero percentage unless the test is perfect. So if we only test meth addicts for meth, our false positive rate will be lower then if we test the population at large.
- jsprogrammer 10y agoI think that is true, but we can still distinguish between the false positive rate of the testing mechanism vs. the false positive rate of a particular series of tests (where this would depend, somewhat, in the underlying distribution). I'm not sure what the correct terminology is for this distinction, but I still don't believe Bayes' Theorem would come into play.