4 ms·

How is it not true? Prior probability has nothing to do with the false positive rate. False positive rate is a function of the inaccuracy of the test.

JshWright 10y ago
Dr Carroll's Healthcare Triage series has been linked elsewhere in the thread, but there are a couple videos that are relevant to this discussion (namely, how Bayes' theorem applies to medical testing, and why the prevalence in the population matters (or, more accurately, the probability that the patient has the disease/drug in the first place, which is heavily influenced by the prevalence)) https://www.youtube.com/watch?v=UF1T7KzRnrs https://www.youtube.com/watch?v=UF1T7KzRnrs https://www.youtube.com/watch?v=Ql2jEJ-6e-Y https://www.youtube.com/watch?v=Ql2jEJ-6e-Y
- jsprogrammer 10y ago
  I am familiar with Bayes' theorem. It just doesn't apply in this situation. False positives are a function of the testing method, not a prior probability. If you wanted to know how likely your positive result is to be true, you would use the theorem.
  - deleted 10y ago
    [deleted]
  - traek 10y ago
    The probability of a false positive result = probability of not doing meth * probability of getting a positive test result while not doing meth. P(+ ∩ M') = P(M') * P(+|M'). That pretty clearly depends on the prior probability, P(M).
    - jsprogrammer 10y ago
      You are calculating the probability for a randomly selected individual of the test to have been given a false positive result. That is different from the test's false positive rate, which is just P(+|M') [Where I assume M' represents that the subject had not used meth.] It doesn't matter what the prior probability is of a random person having used meth (or whatever you are testing for); it only matters how likely it is that someone who has not used meth will be given a positive result.
  - M_Mouse 10y ago
    Say you have a test that determines, with some non-perfect level of accuracy, if a person is or has been president of the USA. If that test is performed only on women, the false positive rate will be 100% regardless of accuracy. All positive results would be false positives. If the test is preformed on any man leaving or entering the oval office, then the false positive rate will likely be significantly less then 100%, as there will be a president tested multiple times in that group, and when we test positive on him, it's not a false positive. If we test the population at large, most positive results are going to be false positives, simply because we're unlikely to actually have a former president get tested. But in theory the false positive rate could be slightly less then 100% if we got lucky. As the probability of an actual occurrence in the test group drops to zero, the rate of false positives approaches 100%, regardless of the accuracy of the testing method.
    - jsprogrammer 10y ago
      I think we are talking about slightly different meanings of false positive rate. I think you might be talking about the rate relative to all positive trials. I think I am talking about the rate relative to all trials that don't meet the conditions of the test. So, you survey 100 women and your test tells me that 10 of them have been president, I say that your test has an observed false positive rate of 10%.
      - M_Mouse 10y ago
        You’re correct in regards to terminology; apparently what I was describing is more related to the terms “positive/negative predictive value.” Your definition of false positive rate is the correct one. But I believe my argument is still valid. If we test a group comprised of only former presidents, all positives must be true positives. So our false positive rate will be 0% while our female test set has been observed to have a false positive rate of 10%. This illustrates that the sample set effects the observed false positive rate. As the rate of occurrence of what we are testing for approaches 100% in the sample set, the false positive rate drops to zero. Inversely, as the incidence of what we are testing for in a (sufficiently large) sample set decreases from 100%, the false positive rate will eventually increase to some non-zero percentage unless the test is perfect. So if we only test meth addicts for meth, our false positive rate will be lower then if we test the population at large.

JshWright 10y ago

Dr Carroll's Healthcare Triage series has been linked elsewhere in the thread, but there are a couple videos that are relevant to this discussion (namely, how Bayes' theorem applies to medical testing, and why the prevalence in the population matters (or, more accurately, the probability that the patient has the disease/drug in the first place, which is heavily influenced by the prevalence)) https://www.youtube.com/watch?v=UF1T7KzRnrs https://www.youtube.com/watch?v=UF1T7KzRnrs https://www.youtube.com/watch?v=Ql2jEJ-6e-Y https://www.youtube.com/watch?v=Ql2jEJ-6e-Y

jsprogrammer 10y ago
I am familiar with Bayes' theorem. It just doesn't apply in this situation. False positives are a function of the testing method, not a prior probability. If you wanted to know how likely your positive result is to be true, you would use the theorem.
- deleted 10y ago
  [deleted]
- traek 10y ago
  The probability of a false positive result = probability of not doing meth * probability of getting a positive test result while not doing meth. P(+ ∩ M') = P(M') * P(+|M'). That pretty clearly depends on the prior probability, P(M).
  - jsprogrammer 10y ago
    You are calculating the probability for a randomly selected individual of the test to have been given a false positive result. That is different from the test's false positive rate, which is just P(+|M') [Where I assume M' represents that the subject had not used meth.] It doesn't matter what the prior probability is of a random person having used meth (or whatever you are testing for); it only matters how likely it is that someone who has not used meth will be given a positive result.
- M_Mouse 10y ago
  Say you have a test that determines, with some non-perfect level of accuracy, if a person is or has been president of the USA. If that test is performed only on women, the false positive rate will be 100% regardless of accuracy. All positive results would be false positives. If the test is preformed on any man leaving or entering the oval office, then the false positive rate will likely be significantly less then 100%, as there will be a president tested multiple times in that group, and when we test positive on him, it's not a false positive. If we test the population at large, most positive results are going to be false positives, simply because we're unlikely to actually have a former president get tested. But in theory the false positive rate could be slightly less then 100% if we got lucky. As the probability of an actual occurrence in the test group drops to zero, the rate of false positives approaches 100%, regardless of the accuracy of the testing method.
  - jsprogrammer 10y ago
    I think we are talking about slightly different meanings of false positive rate. I think you might be talking about the rate relative to all positive trials. I think I am talking about the rate relative to all trials that don't meet the conditions of the test. So, you survey 100 women and your test tells me that 10 of them have been president, I say that your test has an observed false positive rate of 10%.
    - M_Mouse 10y ago
      You’re correct in regards to terminology; apparently what I was describing is more related to the terms “positive/negative predictive value.” Your definition of false positive rate is the correct one. But I believe my argument is still valid. If we test a group comprised of only former presidents, all positives must be true positives. So our false positive rate will be 0% while our female test set has been observed to have a false positive rate of 10%. This illustrates that the sample set effects the observed false positive rate. As the rate of occurrence of what we are testing for approaches 100% in the sample set, the false positive rate drops to zero. Inversely, as the incidence of what we are testing for in a (sufficiently large) sample set decreases from 100%, the false positive rate will eventually increase to some non-zero percentage unless the test is perfect. So if we only test meth addicts for meth, our false positive rate will be lower then if we test the population at large.