5 ms·

"Suppose a drug test is 99% sensitive and 99% specific. That is, the test will produce 99% true positive results for drug users and 99% true negative results fo

"Suppose a drug test is 99% sensitive and 99% specific. That is, the test will produce 99% true positive results for drug users and 99% true negative results for non-drug users. Suppose that 0.5% of people are users of the drug. If a randomly selected individual tests positive, what is the probability that he is a user?"[0] 33.2% thanks to Bayes Theorem. If you have a test with 99% accuracy and 1 in 200 people use the drug, someone testing positive more likely than not is not using the drug. If the drug is less popular, that percentage drops further at a fast rate. [0]https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing https://en.wikipedia.org/wiki/Bayes%27_theorem#Drug_testing

phonon 10y ago
Except you are testing people who are specifically under suspicion, not the general population. The concern here is that it does not appear that these tests are anywhere near 99%/99%...maybe it should be good enough to arrest someone on the scene, but a lab test should be the immediate follow-up, EVERY TIME, not "well maybe we'll do it when we go to trial."
- runako 10y ago
  NB: In America, people are often under suspicion simply due to their ethnicity. QED this isn't good enough to arrest people in America.
- akiselev 10y ago
  Based on my experience with the police in Los Angeles and the article, it seems that they are basically just randomly testing the general population, except maybe with a little filter. The heuristics cops use for "suspicion" are often very biased and caught up in socioeconomics or racism. I wouldn't be surprised if their filters reduced the sample pool by a factor of 2 or 3 which I don't think is enough for probabilities that justify evidence "beyond the shadow of a doubt"
  - bryondowd 10y ago
    The legal standard is beyond reasonable doubt, not beyond the shadow of a doubt. It's intentionally left fuzzy to allow prosecution in cases where absolute certainty is impossible, but we as society still need to have a decent chance of prosecuting offenders. We try to minimize the number of innocents who are wrongfully convicted, but there's always going to be some, otherwise there would be entire classes of crimes that would be almost completely unenforceable, due to their nature. That said, this article made me sick. It's obvious that the system is failing here, probably due to a combination of factors, including a lack of understanding of the accuracy of these tests as well as the filtering used to select who and what gets tested. Even if your test was 99.99% accurate, if officers spend all day gathering crumbs from people's cars and submitting them, and you end up with hundreds of millions of innocent samples to thousands of actual drug samples, you wind up with tens of thousands of false positives. So what we really need is corroborating evidence to get a sample set more skewed toward actual positives. Or just give up and stop trying to enforce possession laws.
    - sixothree 10y ago
      I don't consider it reasonable that 1 in 100, 1 in 10000 people would falsely end up in jail.
- Jtsummers 10y ago
  "Under suspicion"? A traffic stop is sufficient to garner suspicion. Not looking high, not acting high, just being stopped for any ordinary traffic violation is enough.
- michaelmrose 10y ago
  If the effect of getting arrested can in and of itself be life destroying then perhaps its far more rational to keep the evidence for further testing and act if and only a real lab finds the evidence compelling. If the offender is from out of town and skips that is unfortunate but not as unfortunate as wrecking peoples lives.
TheCoelacanth 10y ago
Exactly. Someone not well informed of probability will often assume that it is 99%[0] but the actual probability can be much lower than that when the actual incidence of drug use is low. [0] https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy
mikeash 10y ago
Looking at it the other way, even if you have a 99% chance that the positive result is correct, is a 1% rate of sending innocent people to jail really acceptable?
- LoSboccacc 10y ago
  Or with some context if you put the usa population under test you'd get 3 MILLION innocent convicts. small percentages adds up scary fast over continents
- justicezyx 10y ago
  Other factors should be considered. For example, it is not a random sampling. Polices target those who are statistically more likely to be positive, according to their empirical or historical statistics.
  - mikeash 10y ago
    I think that's already accounted for in my hpothetical, as those are factors that might bring a 33.2% rate up to 99%.
  - bryondowd 10y ago
    Depends how good of a filter the police are using. Even if they are only targeting people who are 10 times as likely to be users as the general population in the above example, that only brings the accuracy to about 84%. And I'd wager their filtering isn't that good. What exactly are they using to determine who is a likely user? Are those people really more likely at all, and if so, how much? I expect it would be highly optimistic to even assume a person targeted by police is even three times as likely as the general population to be in possession of drugs, which would only be about 60%
    - analog31 10y ago
      Indeed, the person could be less likely. From what I've read, affluent whites have the highest rates of illegal drug use, but the police direct their enforcement efforts towards blacks.
forrestthewoods 10y ago
I wonder how statistically accurate USADA drug tests are. What are the odds of a false positive?
- nl 10y ago
  Very low. A second sample is kept and tested separately in the event of a positive. I'm not aware of any cases of false positives because of test errors, and there have been many very well funded challenges.
gaur 10y ago
It's amazing how often a simple Venn diagram or probability tree can cut through the fog and deliver a clear answer to a statistical question. This is slightly off topic, but I can never remember what "specificity" and "sensitivity" are supposed to mean in this context. I wish people (Wikipedia editors included) would stop using these opaque words, and instead just say in plain English what probabilities they're referring to. It's almost as bad as "type 1 error" and "type 2 error", which are terms that are actively hostile to building an understanding of statistical inference.
- coaxial 10y ago
  I find this website[1] pretty good for explaining maths and statistics concepts in a much more accessible way. [1] http://betterexplained.com/articles/an-intuitive-and-short-explanation-of-bayes-theorem/ http://betterexplained.com/articles/an-intuitive-and-short-e...
- k7wqcyfgmu 10y ago
  I find "sensitive" and "specific" to be very clear descriptors. If the test is not specific, it will detect things that are not specifically the drug. If the test is not sensitive enough, it might miss the drug.
coaxial 10y ago
This is very interesting and important to understand. I'm wondering how often has this been successfully argued? Surely a lawyer would have heard about this effect and be able to get the evidence thrown away given the flawed accuracy premise? With this rationale, I suppose lawyers would also advise their clients to not enter a plea bargain agreement? Genuinely curious, hoping someone could shed light on this.
bobwaycott 10y ago
But it says 99%! That means it must be true because there's only a 1% chance it's not true! /s

phonon 10y ago

Except you are testing people who are specifically under suspicion, not the general population. The concern here is that it does not appear that these tests are anywhere near 99%/99%...maybe it should be good enough to arrest someone on the scene, but a lab test should be the immediate follow-up, EVERY TIME, not "well maybe we'll do it when we go to trial."

runako 10y ago
NB: In America, people are often under suspicion simply due to their ethnicity. QED this isn't good enough to arrest people in America.
akiselev 10y ago
Based on my experience with the police in Los Angeles and the article, it seems that they are basically just randomly testing the general population, except maybe with a little filter. The heuristics cops use for "suspicion" are often very biased and caught up in socioeconomics or racism. I wouldn't be surprised if their filters reduced the sample pool by a factor of 2 or 3 which I don't think is enough for probabilities that justify evidence "beyond the shadow of a doubt"
- bryondowd 10y ago
  The legal standard is beyond reasonable doubt, not beyond the shadow of a doubt. It's intentionally left fuzzy to allow prosecution in cases where absolute certainty is impossible, but we as society still need to have a decent chance of prosecuting offenders. We try to minimize the number of innocents who are wrongfully convicted, but there's always going to be some, otherwise there would be entire classes of crimes that would be almost completely unenforceable, due to their nature. That said, this article made me sick. It's obvious that the system is failing here, probably due to a combination of factors, including a lack of understanding of the accuracy of these tests as well as the filtering used to select who and what gets tested. Even if your test was 99.99% accurate, if officers spend all day gathering crumbs from people's cars and submitting them, and you end up with hundreds of millions of innocent samples to thousands of actual drug samples, you wind up with tens of thousands of false positives. So what we really need is corroborating evidence to get a sample set more skewed toward actual positives. Or just give up and stop trying to enforce possession laws.
  - sixothree 10y ago
    I don't consider it reasonable that 1 in 100, 1 in 10000 people would falsely end up in jail.
Jtsummers 10y ago
"Under suspicion"? A traffic stop is sufficient to garner suspicion. Not looking high, not acting high, just being stopped for any ordinary traffic violation is enough.
michaelmrose 10y ago
If the effect of getting arrested can in and of itself be life destroying then perhaps its far more rational to keep the evidence for further testing and act if and only a real lab finds the evidence compelling. If the offender is from out of town and skips that is unfortunate but not as unfortunate as wrecking peoples lives.

TheCoelacanth 10y ago

Exactly. Someone not well informed of probability will often assume that it is 99%[0] but the actual probability can be much lower than that when the actual incidence of drug use is low. [0] https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy https://en.wikipedia.org/wiki/Prosecutor%27s_fallacy

mikeash 10y ago

Looking at it the other way, even if you have a 99% chance that the positive result is correct, is a 1% rate of sending innocent people to jail really acceptable?

LoSboccacc 10y ago
Or with some context if you put the usa population under test you'd get 3 MILLION innocent convicts. small percentages adds up scary fast over continents
justicezyx 10y ago
Other factors should be considered. For example, it is not a random sampling. Polices target those who are statistically more likely to be positive, according to their empirical or historical statistics.
- mikeash 10y ago
  I think that's already accounted for in my hpothetical, as those are factors that might bring a 33.2% rate up to 99%.
- bryondowd 10y ago
  Depends how good of a filter the police are using. Even if they are only targeting people who are 10 times as likely to be users as the general population in the above example, that only brings the accuracy to about 84%. And I'd wager their filtering isn't that good. What exactly are they using to determine who is a likely user? Are those people really more likely at all, and if so, how much? I expect it would be highly optimistic to even assume a person targeted by police is even three times as likely as the general population to be in possession of drugs, which would only be about 60%
  - analog31 10y ago
    Indeed, the person could be less likely. From what I've read, affluent whites have the highest rates of illegal drug use, but the police direct their enforcement efforts towards blacks.

forrestthewoods 10y ago

I wonder how statistically accurate USADA drug tests are. What are the odds of a false positive?

nl 10y ago
Very low. A second sample is kept and tested separately in the event of a positive. I'm not aware of any cases of false positives because of test errors, and there have been many very well funded challenges.

gaur 10y ago

It's amazing how often a simple Venn diagram or probability tree can cut through the fog and deliver a clear answer to a statistical question. This is slightly off topic, but I can never remember what "specificity" and "sensitivity" are supposed to mean in this context. I wish people (Wikipedia editors included) would stop using these opaque words, and instead just say in plain English what probabilities they're referring to. It's almost as bad as "type 1 error" and "type 2 error", which are terms that are actively hostile to building an understanding of statistical inference.

coaxial 10y ago
I find this website[1] pretty good for explaining maths and statistics concepts in a much more accessible way. [1] http://betterexplained.com/articles/an-intuitive-and-short-explanation-of-bayes-theorem/ http://betterexplained.com/articles/an-intuitive-and-short-e...
k7wqcyfgmu 10y ago
I find "sensitive" and "specific" to be very clear descriptors. If the test is not specific, it will detect things that are not specifically the drug. If the test is not sensitive enough, it might miss the drug.

coaxial 10y ago

This is very interesting and important to understand. I'm wondering how often has this been successfully argued? Surely a lawyer would have heard about this effect and be able to get the evidence thrown away given the flawed accuracy premise? With this rationale, I suppose lawyers would also advise their clients to not enter a plea bargain agreement? Genuinely curious, hoping someone could shed light on this.

bobwaycott 10y ago

But it says 99%! That means it must be true because there's only a 1% chance it's not true! /s