4 ms·

This is a pretty bad example to illustrate the fallacy because a 25% confidence is actually extremely good. I don't know what good a test for sociopathy is, but

This is a pretty bad example to illustrate the fallacy because a 25% confidence is actually extremely good. I don't know what good a test for sociopathy is, but if we had a test this good at identifying terrorists, it would be incredibly useful. If signals intelligence could produce a list of people and guarantee that a quarter of the people on that list are terrorists, it would absolutely revolutionize law enforcement.

paganel 13y ago
> and guarantee that a quarter of the people on that list are terrorists, Honest question, what would happen to the other three quarters?
- brazzy 13y ago
  They'd be subject to background checks and surveillance that they ideally would not even notice - yes, I'm aware that we don't live in an ideal world, but the point is that traditional "leg work" policing is pretty good at determining whether a suspect is actually engaged in nefarious activities - but it's expensive and requires a reasonably narrow list of suspects to begin with.
- anigbrowl 13y ago
  Most likely nothing. It depends on what sort of test it is; if it's something non-intuitive like (say) a habit of writing sentences that always have a prime number of words in them, you'll get your false positives but most those people won't pass any other tests, whereas the actual terrorists will. What Schneier is missing is that while you can't ID people that well from a single test, you can apply a bunch of them. In his example, one test improves the probability of correctly ID a sociopath from 4% to 24%. Apply another, different test of similar efficacy to that result set and you'll have a population of 21 true positives, and 8 or 9 false positives, increasing the probabiliy of a successful ID from 25% to ~70%. Sure, there's no single test that will give you reliable answers, but so what? It's OK to use a multi-pronged solution.
  - bandushrew 13y ago
    Waiting until someone actually commits a crime provides a list that is 100% accurate.
    - anigbrowl 13y ago
      No, it doesn't. People commit crimes and get away with them al the time, in fact. I'm not proposing that we put people in jail for having criminal potential.
      - bandushrew 13y ago
        Nothing that is being proposed will stop people getting away with crimes. Waiting until someone actually commits a crime will stop people being persecuted for a coincidental similarity of their behavior to that of a terrorist, sociopath or mime artist.
    - finnw 13y ago
      I can just see the Daily Mail headline now: "Facebook records reveal convicted killer wrote 13-word post 5 years ago - red flag was raised - why was nothing done?"
  - fpgeek 13y ago
    Applying multiple tests only works if the tests are independent. When you're searching for the proverbial needle in the haystack, you probably don't have enough needles to let you reliably calibrate several independent tests in the first place.
- jkldotio 13y ago
  They'd end up on a no fly list and be deprived of basic liberties to travel with no right to appeal secret courts and determinations. Or they'd get invasive checks even though they are a kid, or in a collapsible wheelchair and obviously not a terrorist. All because people can't understand the example in question, which appears in the first few chapters of most introduction to statistics books. And while all that money is being spent on useless checks the 9/11 terrorists, who the agencies were warned about, and the Boston bombers, who the agencies were ALSO warned about, are not followed up on because human and other resources are being spent on mass surveillance.
smackay 13y ago
This issue has nothing to do with the efficacy of the tests. In a perfect world having a list where 25% of the names are potential trouble-makers (for various definitions of trouble-maker) would be an enormous benefit and allow resources to be targeted more efficiently. The real problem is what happens in the imperfect world where law enforcement, government, self-imposed officers of authority get lazy or down-right malevolent. Do you really want to live in a world where your family members might be carted off, never to return, because their name came up on a list generated by a computer. Try asking the folks in North Korea, for example, whether this type of test is a good idea or not.
- 7952 13y ago
  The best outcome would be for these kind of traits to be passively detected and for the community to provide help and support as an emergent property of that community. In a way this already happens on sites like Reddit where suicide prevention emerged organically based on the dynamics of the community. This is in stark contrast to the "real world". The internet is well known as a negative influence on certain people, but couldn't it be having a positive effect that is harder to measure and more an unintended side effect.
  - smackay 13y ago
    Yes, but what happens when your community find out that you no longer believe in God or that you think the Earth actually revolves around the Sun. It's not when things go right but when things go wrong that matter. The points of failure for this type of system are innumerable. The real goal is building social/political systems that are robust and have checks and balances so that they cannot be perverted by special interests and are accessible to those who need them (child abuse support lines are a good example). Anything where a group intervenes on behalf of an individual is prone to disaster.
    - 7952 13y ago
      The system you describe is exactly what we have in the world now. The financial system has numerous checks and balances and is notoriously prone to non-virtuous behavior. Aren't hacker news or stackexchange examples of creating virtuous behavior using an algorithm, and a strong community? It is self regulating because if you suddenly have a deep opposition to the ethos of a community you can just leave (slashdot -> digg -> reddit).
ZeroGravitas 13y ago
I think the fact that in this hypothetical 4% of all people are actually terrorists, and the potential terrorist list would be 10% of the total population, would have a greater impact on law enforcement than the accuracy of the test. Even 0.4% of the population that get tested and are incorrectly "proved" innocent of being a terrorist amounts to more than a million undetectable terrorists in the US alone.
mortov 13y ago
Do I understand you correctly ? You're saying that you would be happy to join 74 other non-terrorists (i.e. law abiding citizens) plus 25 actual terrorists and be taken off to Guantanamo Bay indefinitely ? You're really sure about that being a Good Thing for law enforcement ?
abecedarius 13y ago
He got 25% starting from a base rate of 4%. The base rate for terrorists is a little lower than that. I agree that the essay probably ought to emphasize this point so skimmers won't take away the wrong idea.
stephencanon 13y ago
I don’t pretend to know what the value of N ought to be[1], but 1/3 is ridiculously low. [1] http://www2.law.ucla.edu/volokh/guilty.htm http://www2.law.ucla.edu/volokh/guilty.htm

paganel 13y ago

> and guarantee that a quarter of the people on that list are terrorists, Honest question, what would happen to the other three quarters?

brazzy 13y ago
They'd be subject to background checks and surveillance that they ideally would not even notice - yes, I'm aware that we don't live in an ideal world, but the point is that traditional "leg work" policing is pretty good at determining whether a suspect is actually engaged in nefarious activities - but it's expensive and requires a reasonably narrow list of suspects to begin with.
anigbrowl 13y ago
Most likely nothing. It depends on what sort of test it is; if it's something non-intuitive like (say) a habit of writing sentences that always have a prime number of words in them, you'll get your false positives but most those people won't pass any other tests, whereas the actual terrorists will. What Schneier is missing is that while you can't ID people that well from a single test, you can apply a bunch of them. In his example, one test improves the probability of correctly ID a sociopath from 4% to 24%. Apply another, different test of similar efficacy to that result set and you'll have a population of 21 true positives, and 8 or 9 false positives, increasing the probabiliy of a successful ID from 25% to ~70%. Sure, there's no single test that will give you reliable answers, but so what? It's OK to use a multi-pronged solution.
- bandushrew 13y ago
  Waiting until someone actually commits a crime provides a list that is 100% accurate.
  - anigbrowl 13y ago
    No, it doesn't. People commit crimes and get away with them al the time, in fact. I'm not proposing that we put people in jail for having criminal potential.
    - bandushrew 13y ago
      Nothing that is being proposed will stop people getting away with crimes. Waiting until someone actually commits a crime will stop people being persecuted for a coincidental similarity of their behavior to that of a terrorist, sociopath or mime artist.
  - finnw 13y ago
    I can just see the Daily Mail headline now: "Facebook records reveal convicted killer wrote 13-word post 5 years ago - red flag was raised - why was nothing done?"
- fpgeek 13y ago
  Applying multiple tests only works if the tests are independent. When you're searching for the proverbial needle in the haystack, you probably don't have enough needles to let you reliably calibrate several independent tests in the first place.
jkldotio 13y ago
They'd end up on a no fly list and be deprived of basic liberties to travel with no right to appeal secret courts and determinations. Or they'd get invasive checks even though they are a kid, or in a collapsible wheelchair and obviously not a terrorist. All because people can't understand the example in question, which appears in the first few chapters of most introduction to statistics books. And while all that money is being spent on useless checks the 9/11 terrorists, who the agencies were warned about, and the Boston bombers, who the agencies were ALSO warned about, are not followed up on because human and other resources are being spent on mass surveillance.

smackay 13y ago

This issue has nothing to do with the efficacy of the tests. In a perfect world having a list where 25% of the names are potential trouble-makers (for various definitions of trouble-maker) would be an enormous benefit and allow resources to be targeted more efficiently. The real problem is what happens in the imperfect world where law enforcement, government, self-imposed officers of authority get lazy or down-right malevolent. Do you really want to live in a world where your family members might be carted off, never to return, because their name came up on a list generated by a computer. Try asking the folks in North Korea, for example, whether this type of test is a good idea or not.

7952 13y ago
The best outcome would be for these kind of traits to be passively detected and for the community to provide help and support as an emergent property of that community. In a way this already happens on sites like Reddit where suicide prevention emerged organically based on the dynamics of the community. This is in stark contrast to the "real world". The internet is well known as a negative influence on certain people, but couldn't it be having a positive effect that is harder to measure and more an unintended side effect.
- smackay 13y ago
  Yes, but what happens when your community find out that you no longer believe in God or that you think the Earth actually revolves around the Sun. It's not when things go right but when things go wrong that matter. The points of failure for this type of system are innumerable. The real goal is building social/political systems that are robust and have checks and balances so that they cannot be perverted by special interests and are accessible to those who need them (child abuse support lines are a good example). Anything where a group intervenes on behalf of an individual is prone to disaster.
  - 7952 13y ago
    The system you describe is exactly what we have in the world now. The financial system has numerous checks and balances and is notoriously prone to non-virtuous behavior. Aren't hacker news or stackexchange examples of creating virtuous behavior using an algorithm, and a strong community? It is self regulating because if you suddenly have a deep opposition to the ethos of a community you can just leave (slashdot -> digg -> reddit).

ZeroGravitas 13y ago

I think the fact that in this hypothetical 4% of all people are actually terrorists, and the potential terrorist list would be 10% of the total population, would have a greater impact on law enforcement than the accuracy of the test. Even 0.4% of the population that get tested and are incorrectly "proved" innocent of being a terrorist amounts to more than a million undetectable terrorists in the US alone.

mortov 13y ago

Do I understand you correctly ? You're saying that you would be happy to join 74 other non-terrorists (i.e. law abiding citizens) plus 25 actual terrorists and be taken off to Guantanamo Bay indefinitely ? You're really sure about that being a Good Thing for law enforcement ?

abecedarius 13y ago

He got 25% starting from a base rate of 4%. The base rate for terrorists is a little lower than that. I agree that the essay probably ought to emphasize this point so skimmers won't take away the wrong idea.

stephencanon 13y ago

I don’t pretend to know what the value of N ought to be[1], but 1/3 is ridiculously low. [1] http://www2.law.ucla.edu/volokh/guilty.htm http://www2.law.ucla.edu/volokh/guilty.htm