5 ms·

Their method had a 89.5% success rate, which might seem great, but is pretty much worthless in real life. The US has the highest incarceration rate, so we can

by decker 10y ago

Their method had a 89.5% success rate, which might seem great, but is pretty much worthless in real life. The US has the highest incarceration rate, so we can use the US incarceration rate as an upper bound for the probability of randomly selecting a criminal from the population (716 per 100k, P=0.00716). This means that if we apply the same method at random to members of the general population, there's actually at most a 5.79% chance that the result of "criminal" is accurate. Maths: Let Pc = probability of criminal = 0.00716 Let Pt = probability of test being accurate = .895 Probability of criminal given criminal conclusion = Pc * Pt / (Pc * Pt + (1 - Pc) * (1 - Pt)) = 0.0579

dx034 10y ago
Thanks for calculating this, it's astonishing how few journalists can do this math. It shows how unreliable networks currently are. Any model you'd want to use on real world population would need to have detection rates >99.9%, even if you're just talking about pre-screening.
- petra 10y ago
  >> it's astonishing how few journalists can do this math. Good journalists usually let 3rd party experts chime in(this news site usually does that), which seems like a useful filter for news quality.
fractalwrench 10y ago
If the method was specifically targeted at high-crime neighbourhoods, the chance of an accurate match would presumably increase. You could also map the entire population of a country and detect previously unknown "criminal" hotspots. Both are awful, authoritarian ideas, but would potentially give useful information to support regular policing. Thankfully, the most likely application of this technology I can see in the near-future is someone making an app that scores your face for criminality.
- tremon 10y ago
  Thankfully, the most likely application of this technology I can see in the near-future is someone making an app that scores your face for criminality. I don't share your positive outlook on law enforcement officials.
- EliRivers 10y ago
  This will then lead to people failing job interviews because the app didn't like their face.
soVeryTired 10y ago
Their data had a 50/50 split between criminals and non-criminals, so an 89.5% success rate is actually pretty good. You're right that this algorithm wouldn't be practical to detect criminals in the general population (ethical issues notwithstanding), but I don't think they're suggesting that. They're just offering a neat statistical explanation for a 2011 study that showed that under certain controlled conditions, people are surprisingly good at spotting convicts.
YeGoblynQueenne 10y ago
>> Probability of criminal given criminal conclusion That's fudging things a bit, isn't it? The 89.5% is the probability a criminal conclusion is accurate, not the probability of the system reaching a criminal conclusion itself.

by decker 10y ago

dx034 10y ago

Thanks for calculating this, it's astonishing how few journalists can do this math. It shows how unreliable networks currently are. Any model you'd want to use on real world population would need to have detection rates >99.9%, even if you're just talking about pre-screening.

petra 10y ago
>> it's astonishing how few journalists can do this math. Good journalists usually let 3rd party experts chime in(this news site usually does that), which seems like a useful filter for news quality.

fractalwrench 10y ago

If the method was specifically targeted at high-crime neighbourhoods, the chance of an accurate match would presumably increase. You could also map the entire population of a country and detect previously unknown "criminal" hotspots. Both are awful, authoritarian ideas, but would potentially give useful information to support regular policing. Thankfully, the most likely application of this technology I can see in the near-future is someone making an app that scores your face for criminality.

tremon 10y ago
Thankfully, the most likely application of this technology I can see in the near-future is someone making an app that scores your face for criminality. I don't share your positive outlook on law enforcement officials.
EliRivers 10y ago
This will then lead to people failing job interviews because the app didn't like their face.

soVeryTired 10y ago

Their data had a 50/50 split between criminals and non-criminals, so an 89.5% success rate is actually pretty good. You're right that this algorithm wouldn't be practical to detect criminals in the general population (ethical issues notwithstanding), but I don't think they're suggesting that. They're just offering a neat statistical explanation for a 2011 study that showed that under certain controlled conditions, people are surprisingly good at spotting convicts.

YeGoblynQueenne 10y ago

>> Probability of criminal given criminal conclusion That's fudging things a bit, isn't it? The 89.5% is the probability a criminal conclusion is accurate, not the probability of the system reaching a criminal conclusion itself.