9 ms·

Neural Net Trained on Mugshots Predicts Criminals

68 points by jastr 10y ago

grzm 10y ago
Previous discussion on arXiv paper: https://news.ycombinator.com/item?id=12983827 https://news.ycombinator.com/item?id=12983827
- mc32 10y ago
  So if the 2011 paper shows similar conclusions as this paper... and if people as well as this CNN can "tell" criminals apart from non-criminals could non-criminal people's unconscious bias lead to non-criminal faced people to see criminal-faced as actual criminals and kind of funnel them into criminality? That is non-criminal faced people might show bias against criminal faced people such that these criminal-faced people resort to [petty or other] crime to get by? Of course, one question is why this bias might have emerged in the first place. What was its genesis?
jbpetersen 10y ago
"Their method is straightforward. They take ID photos of 1856 Chinese men between the ages of 18 and 55 with no facial hair. Half of these men were criminals. They then used 90 percent of these images to train a convolutional neural network to recognize the difference and then tested the neural net on the remaining 10 percent of the images. The results are unsettling. Xiaolin and Xi found that the neural network could correctly identify criminals and noncriminals with an accuracy of 89.5 percent."
- deleted 10y ago
  [deleted]
- rdlecler1 10y ago
  I'd like to see what human recognition would be as a null hypothesis. You could also block out certain features to see what's driving the decision making here.
  - jbpetersen 10y ago
    The article details the specific facial features that were focused on by the trained model. So that's at least part of what you're looking for.
- ganeshkrishnan 10y ago
  89.5 accuracy is real bad. It looks pretty convincing but the probability of a person being a criminal turns out to be pretty low. Cancer detecting machines have probability close to 99% but even if it says you have cancer the probability is not that high (close to 10% I think) because of false positives. Likewise if this network predicts a user to be criminal when he is not then the probability of success goes down.
KKKKkkkk1 10y ago
I bet you could train a neural net to detect the exact moment when the AI bubble has jumped the shark.
- foota 10y ago
  Right when you turn on the shark jump detecting AI, would be my bet.
- spacemanmatt 10y ago
  What with the breakthroughs in Fonzi-recognition lately I think we could be looking at a full-on shark jump detector by mid-2017
woodruffw 10y ago
Give this to a despot to train on his political enemies (or ethnic/religious minorities), and you suddenly have a very good NN for condemning innocent people to jail. Research should continue into this, but it's worth remembering that the "criminals" being trained on aren't necessarily bad people in the moral sense. They're merely the recipients of judgment by some third entity (in this case, China's legal system).
- c0nducktr 10y ago
  Phrenology for the 21st century.
thechao 10y ago
Alternately: criminal prosecution is targeted at people who "look like" criminals; thus, the NN is just selecting those people we think look like criminals, rather than any inherent criminality.
- ganeshkrishnan 10y ago
  This is so true. We are training the neural networks from our own bias and cannot expect it to be fair. A supervised ML algorithm is as good as it's training data and in our current scenario the training data is almost always flawed. Judges receiving kickbacks to throw minors into juvenile jail, racial bias in sentences, jail for marijuana, private prison etc. These are all trademarks of a broken penal system and we cannot and should not train machines based on this data.
  - daveguy 10y ago
    > Judges receiving kickbacks to throw minors into juvenile jail, racial bias in sentences, jail for marijuana, private prison etc. I expect you have some solid evidence for this? Maybe just a citation? Anything? Obviously there is institutional racial bias, but kickbacks for private jails? for marijuana? for minors? Really?
    - ken_railey 10y ago
      Not the person you're responding to, but I assume that the kickbacks comment was referring to cases like https://en.wikipedia.org/wiki/Kids_for_cash_scandal https://en.wikipedia.org/wiki/Kids_for_cash_scandal
    - woodruffw 10y ago
      > Judges receiving kickbacks to throw minors into juvenile jail He's probably referring to "Kids For Cash" [0]. > racial bias in sentences You can look at federal and state incarceration statistics for yourself. African-Americans and Hispanic-Americans serve, on average, longer sentences for the same crimes as their Caucasian counterparts. Here's one source.[1] > jail for marijuana Mandatory minimums for possession (especially "with intent to sell") have been part of federal drug enforcement policy for the last 50 years. Here are a few of them.[2] > private prison etc It was only this year that the DOJ finally committed to eliminating private prisons in the federal system.[3] They're still a massive part of state penitentiary systems (see: cash for kids). [0]: https://en.wikipedia.org/wiki/Kids_for_cash_scandal https://en.wikipedia.org/wiki/Kids_for_cash_scandal [1]: https://www.aclu.org/issues/mass-incarceration/racial-disparities-criminal-justice https://www.aclu.org/issues/mass-incarceration/racial-dispar... [2]: http://www.pbs.org/wgbh/pages/frontline/shows/snitch/primer/ http://www.pbs.org/wgbh/pages/frontline/shows/snitch/primer/ [3]: https://www.justice.gov/opa/blog/phasing-out-our-use-private-prisons https://www.justice.gov/opa/blog/phasing-out-our-use-private...
empath75 10y ago
'Criminals' being defined as people who have been convicted by a judicial system that is full of bias.
- freyr 10y ago
  What is the bias in this case?
  - null0pointer 10y ago
    Every country has different laws so a 'criminal' in one jurisdiction may not be a 'criminal' in another. The training data is fundamentally flawed as the target classes (criminal and not-criminal) are defined by largely arbitrary rules.
    - freyr 10y ago
      We can simply restrict our attention to one country, as they did in the study, to eliminate that variability. What is the bias?
okonomiyaki3000 10y ago
This concept is troubling, uncomfortable, and could potentially be the basis for some very bad policy but none of that is a reason to dismiss it outright. If these correlations are real, it's worthwhile to find out more about it with an open mind.
- Ace17 10y ago
  Minority report scenario hinges on the fact that the crime predictions are not completely reliable. But if we had a 100% sure way of detecting that someone's going to commit a crime, should we use it? This would be a dangerous change of perspective about free will and its relation to the law.
  - SomeStupidPoint 10y ago
    Also worth noting: predicting that someone will commit a crime doesn't mean that we have to arrest them. We could use the prediction to target other (voluntary) intervention. Of course, we already use less than 100% effective prediction to target non-arrest intervention, so it's quite possible that 100% prediction would only have the effect to better target what we already do (and potentially alleviate harm or waste), rather than Minority Report style pre-crime arrests. Im not sure crime prediction is a moral quagmire, just that what we use as a basis for prediction or a few drastic uses, like pre-crime arrests, are tough questions.
  - halomru 10y ago
    >if we had a 100% sure way of detecting that someone's going to commit a crime, should we use it If it was 100% correct, it seems morally justifiable to use it, as long as you are a dictatorship. For a democracy, you would need not only correctness, but independently verifiable correctness. If the algorithm is 100% correct today but can be influenced tomorrow, you are giving absolute power over the state to whoever can influence the algorithm. After all, it's impossible to proof your innocence if you are convicted before the crime. Basically, after solving the technical problems with creating the algorithm, you also need a system of checks and balances that can replace the public verifiability of our current court system.
sundvor 10y ago
So were faces of e.g. dodgy GFC bankers included in that pool? Or are we just talking petty criminals here.
deleted 10y ago
[deleted]
rsl7 10y ago
I wonder if it is the same before and after one becomes a criminal. Being a criminal does not mean they don't regret or feel guilt. Perhaps that is what is detected.
jlos 10y ago
Did not think this would be the year phrenology made a comeback. . .
- tropo 10y ago
  Why not? The inaccuracy problems were merely poor-quality measurements and statistics. Prediction accuracy will continue to improve with technology.
sdoering 10y ago
Well what comes around, goes around or such. I remember during my studies (Literature, Culture and such) to having read of methodologies used in the 18th century to detect criminals by their physiological features. On person doing these studies was for example Francis Galton (who by the way did quite a mix of things from eugenics to statistics and "the wisdom of the crowd")[1]. Lots of things, coming from the old times into the modern times like Physignomy [2] that was also used for racial identification/discrimination in the 20th century. Have fun walking deeper into that rabbit hole of history. What I take from that is, that bad ideas never die, even if science was able to debunk them. Or as @thechao already said: > Alternately: criminal prosecution is targeted at people who "look like" criminals; thus, the NN is just selecting those people we think look like criminals, rather than any inherent criminality. [1] https://en.wikipedia.org/wiki/Francis_Galton https://en.wikipedia.org/wiki/Francis_Galton [2] https://en.wikipedia.org/wiki/Physiognomy https://en.wikipedia.org/wiki/Physiognomy [Edit] Formatting
irascible 10y ago
No discussion of where these datasets came from? They're probably training the nn to recognize mugshots vs normal shot. -10 pts for research being conducted by the chinese. -10 pts for this "research" coming out right as the us is entering a new fascist regime.. I call bs.. not to mention other excellent points raised in this thread...
decker 10y ago
Their method had a 89.5% success rate, which might seem great, but is pretty much worthless in real life. The US has the highest incarceration rate, so we can use the US incarceration rate as an upper bound for the probability of randomly selecting a criminal from the population (716 per 100k, P=0.00716). This means that if we apply the same method at random to members of the general population, there's actually at most a 5.79% chance that the result of "criminal" is accurate. Maths: Let Pc = probability of criminal = 0.00716 Let Pt = probability of test being accurate = .895 Probability of criminal given criminal conclusion = Pc * Pt / (Pc * Pt + (1 - Pc) * (1 - Pt)) = 0.0579
- dx034 10y ago
  Thanks for calculating this, it's astonishing how few journalists can do this math. It shows how unreliable networks currently are. Any model you'd want to use on real world population would need to have detection rates >99.9%, even if you're just talking about pre-screening.
  - petra 10y ago
    >> it's astonishing how few journalists can do this math. Good journalists usually let 3rd party experts chime in(this news site usually does that), which seems like a useful filter for news quality.
- fractalwrench 10y ago
  If the method was specifically targeted at high-crime neighbourhoods, the chance of an accurate match would presumably increase. You could also map the entire population of a country and detect previously unknown "criminal" hotspots. Both are awful, authoritarian ideas, but would potentially give useful information to support regular policing. Thankfully, the most likely application of this technology I can see in the near-future is someone making an app that scores your face for criminality.
  - tremon 10y ago
    Thankfully, the most likely application of this technology I can see in the near-future is someone making an app that scores your face for criminality. I don't share your positive outlook on law enforcement officials.
aktiur 10y ago
"They take ID photos of 1856 Chinese men [...]. Half of these men were criminals." Because half of the general male population is criminal, of course. The accuracy rate would be very different with a training/testing sample that takes the base rate of criminality into account.
KaiserPro 10y ago
Phrenology is still phrenology even if its digital.
paulajohnson 10y ago
This doesn't "predict criminals", it predicts those who will be convicted of a crime, which is not the same thing. Suppose that people have an unconscious prejudice against those with eyes set close together. They will be disproportionately convicted, and this neural net will find the correlation.
- bryanrasmussen 10y ago
  Since people assume beautiful people are good what is the correlation between these results and people who are considered ugly in their particular population?
stewhuk 10y ago
I looked at this paper the other day and it looked to me like the non-criminal faces examples were men with shirt collars, whereas the criminal examples were men in t-shirts. If they got above random accuracy, then I wonder if they simply overfitted on that, and the lighting and colour differences between the two styles of photos.
mobiuscog 10y ago
The future of this would make a great film...
YeGoblynQueenne 10y ago
The paper's conclusion claims that " Furthermore, we have discovered that a law of normality for faces of non- criminals. After controlled for race, gender and age, the general law-biding public have facial appearances that vary in a significantly lesser degree than criminals." Given the above I move that the title of this post is changed to "Neural Net trained on mugshots confirms the findings of Phrenology".
spacemanmatt 10y ago
Could it be used to identify those likely to commit election or securities frauds? Or maybe those likely to poison an entire city by ruining their water supply?
stolk 10y ago
Let me guess... the NN was trained to spot tattoos? I bet you tattoos and crime correlate.
kuroguro 10y ago
"In other words, the faces of general law-biding public have a greater degree of resemblance compared with the faces of criminals, or criminals have a higher degree of dissimilarity in facial appearance than normal people" Hmm... so. You look weird -> people treat you worse -> you don't feel like working with them -> higher chance of being a criminal. I know that's a giant leap in reasoning, but that was the first thing that came to mind.

68 points by jastr 10y ago

grzm 10y ago

Previous discussion on arXiv paper: https://news.ycombinator.com/item?id=12983827 https://news.ycombinator.com/item?id=12983827

mc32 10y ago
So if the 2011 paper shows similar conclusions as this paper... and if people as well as this CNN can "tell" criminals apart from non-criminals could non-criminal people's unconscious bias lead to non-criminal faced people to see criminal-faced as actual criminals and kind of funnel them into criminality? That is non-criminal faced people might show bias against criminal faced people such that these criminal-faced people resort to [petty or other] crime to get by? Of course, one question is why this bias might have emerged in the first place. What was its genesis?

jbpetersen 10y ago

"Their method is straightforward. They take ID photos of 1856 Chinese men between the ages of 18 and 55 with no facial hair. Half of these men were criminals. They then used 90 percent of these images to train a convolutional neural network to recognize the difference and then tested the neural net on the remaining 10 percent of the images. The results are unsettling. Xiaolin and Xi found that the neural network could correctly identify criminals and noncriminals with an accuracy of 89.5 percent."

deleted 10y ago
[deleted]
rdlecler1 10y ago
I'd like to see what human recognition would be as a null hypothesis. You could also block out certain features to see what's driving the decision making here.
- jbpetersen 10y ago
  The article details the specific facial features that were focused on by the trained model. So that's at least part of what you're looking for.
ganeshkrishnan 10y ago
89.5 accuracy is real bad. It looks pretty convincing but the probability of a person being a criminal turns out to be pretty low. Cancer detecting machines have probability close to 99% but even if it says you have cancer the probability is not that high (close to 10% I think) because of false positives. Likewise if this network predicts a user to be criminal when he is not then the probability of success goes down.

KKKKkkkk1 10y ago

I bet you could train a neural net to detect the exact moment when the AI bubble has jumped the shark.

foota 10y ago
Right when you turn on the shark jump detecting AI, would be my bet.
spacemanmatt 10y ago
What with the breakthroughs in Fonzi-recognition lately I think we could be looking at a full-on shark jump detector by mid-2017

woodruffw 10y ago

Give this to a despot to train on his political enemies (or ethnic/religious minorities), and you suddenly have a very good NN for condemning innocent people to jail. Research should continue into this, but it's worth remembering that the "criminals" being trained on aren't necessarily bad people in the moral sense. They're merely the recipients of judgment by some third entity (in this case, China's legal system).

c0nducktr 10y ago
Phrenology for the 21st century.

thechao 10y ago

Alternately: criminal prosecution is targeted at people who "look like" criminals; thus, the NN is just selecting those people we think look like criminals, rather than any inherent criminality.

ganeshkrishnan 10y ago
This is so true. We are training the neural networks from our own bias and cannot expect it to be fair. A supervised ML algorithm is as good as it's training data and in our current scenario the training data is almost always flawed. Judges receiving kickbacks to throw minors into juvenile jail, racial bias in sentences, jail for marijuana, private prison etc. These are all trademarks of a broken penal system and we cannot and should not train machines based on this data.
- daveguy 10y ago
  > Judges receiving kickbacks to throw minors into juvenile jail, racial bias in sentences, jail for marijuana, private prison etc. I expect you have some solid evidence for this? Maybe just a citation? Anything? Obviously there is institutional racial bias, but kickbacks for private jails? for marijuana? for minors? Really?
  - ken_railey 10y ago
    Not the person you're responding to, but I assume that the kickbacks comment was referring to cases like https://en.wikipedia.org/wiki/Kids_for_cash_scandal https://en.wikipedia.org/wiki/Kids_for_cash_scandal
  - woodruffw 10y ago
    > Judges receiving kickbacks to throw minors into juvenile jail He's probably referring to "Kids For Cash" [0]. > racial bias in sentences You can look at federal and state incarceration statistics for yourself. African-Americans and Hispanic-Americans serve, on average, longer sentences for the same crimes as their Caucasian counterparts. Here's one source.[1] > jail for marijuana Mandatory minimums for possession (especially "with intent to sell") have been part of federal drug enforcement policy for the last 50 years. Here are a few of them.[2] > private prison etc It was only this year that the DOJ finally committed to eliminating private prisons in the federal system.[3] They're still a massive part of state penitentiary systems (see: cash for kids). [0]: https://en.wikipedia.org/wiki/Kids_for_cash_scandal https://en.wikipedia.org/wiki/Kids_for_cash_scandal [1]: https://www.aclu.org/issues/mass-incarceration/racial-disparities-criminal-justice https://www.aclu.org/issues/mass-incarceration/racial-dispar... [2]: http://www.pbs.org/wgbh/pages/frontline/shows/snitch/primer/ http://www.pbs.org/wgbh/pages/frontline/shows/snitch/primer/ [3]: https://www.justice.gov/opa/blog/phasing-out-our-use-private-prisons https://www.justice.gov/opa/blog/phasing-out-our-use-private...

empath75 10y ago

'Criminals' being defined as people who have been convicted by a judicial system that is full of bias.

freyr 10y ago
What is the bias in this case?
- null0pointer 10y ago
  Every country has different laws so a 'criminal' in one jurisdiction may not be a 'criminal' in another. The training data is fundamentally flawed as the target classes (criminal and not-criminal) are defined by largely arbitrary rules.
  - freyr 10y ago
    We can simply restrict our attention to one country, as they did in the study, to eliminate that variability. What is the bias?

okonomiyaki3000 10y ago

This concept is troubling, uncomfortable, and could potentially be the basis for some very bad policy but none of that is a reason to dismiss it outright. If these correlations are real, it's worthwhile to find out more about it with an open mind.

Ace17 10y ago
Minority report scenario hinges on the fact that the crime predictions are not completely reliable. But if we had a 100% sure way of detecting that someone's going to commit a crime, should we use it? This would be a dangerous change of perspective about free will and its relation to the law.
- SomeStupidPoint 10y ago
  Also worth noting: predicting that someone will commit a crime doesn't mean that we have to arrest them. We could use the prediction to target other (voluntary) intervention. Of course, we already use less than 100% effective prediction to target non-arrest intervention, so it's quite possible that 100% prediction would only have the effect to better target what we already do (and potentially alleviate harm or waste), rather than Minority Report style pre-crime arrests. Im not sure crime prediction is a moral quagmire, just that what we use as a basis for prediction or a few drastic uses, like pre-crime arrests, are tough questions.
- halomru 10y ago
  >if we had a 100% sure way of detecting that someone's going to commit a crime, should we use it If it was 100% correct, it seems morally justifiable to use it, as long as you are a dictatorship. For a democracy, you would need not only correctness, but independently verifiable correctness. If the algorithm is 100% correct today but can be influenced tomorrow, you are giving absolute power over the state to whoever can influence the algorithm. After all, it's impossible to proof your innocence if you are convicted before the crime. Basically, after solving the technical problems with creating the algorithm, you also need a system of checks and balances that can replace the public verifiability of our current court system.

sundvor 10y ago

So were faces of e.g. dodgy GFC bankers included in that pool? Or are we just talking petty criminals here.

deleted 10y ago

[deleted]

rsl7 10y ago

I wonder if it is the same before and after one becomes a criminal. Being a criminal does not mean they don't regret or feel guilt. Perhaps that is what is detected.

jlos 10y ago

Did not think this would be the year phrenology made a comeback. . .

tropo 10y ago
Why not? The inaccuracy problems were merely poor-quality measurements and statistics. Prediction accuracy will continue to improve with technology.

sdoering 10y ago

Well what comes around, goes around or such. I remember during my studies (Literature, Culture and such) to having read of methodologies used in the 18th century to detect criminals by their physiological features. On person doing these studies was for example Francis Galton (who by the way did quite a mix of things from eugenics to statistics and "the wisdom of the crowd")[1]. Lots of things, coming from the old times into the modern times like Physignomy [2] that was also used for racial identification/discrimination in the 20th century. Have fun walking deeper into that rabbit hole of history. What I take from that is, that bad ideas never die, even if science was able to debunk them. Or as @thechao already said: > Alternately: criminal prosecution is targeted at people who "look like" criminals; thus, the NN is just selecting those people we think look like criminals, rather than any inherent criminality. [1] https://en.wikipedia.org/wiki/Francis_Galton https://en.wikipedia.org/wiki/Francis_Galton [2] https://en.wikipedia.org/wiki/Physiognomy https://en.wikipedia.org/wiki/Physiognomy [Edit] Formatting

irascible 10y ago

No discussion of where these datasets came from? They're probably training the nn to recognize mugshots vs normal shot. -10 pts for research being conducted by the chinese. -10 pts for this "research" coming out right as the us is entering a new fascist regime.. I call bs.. not to mention other excellent points raised in this thread...

decker 10y ago

Their method had a 89.5% success rate, which might seem great, but is pretty much worthless in real life. The US has the highest incarceration rate, so we can use the US incarceration rate as an upper bound for the probability of randomly selecting a criminal from the population (716 per 100k, P=0.00716). This means that if we apply the same method at random to members of the general population, there's actually at most a 5.79% chance that the result of "criminal" is accurate. Maths: Let Pc = probability of criminal = 0.00716 Let Pt = probability of test being accurate = .895 Probability of criminal given criminal conclusion = Pc * Pt / (Pc * Pt + (1 - Pc) * (1 - Pt)) = 0.0579

dx034 10y ago
Thanks for calculating this, it's astonishing how few journalists can do this math. It shows how unreliable networks currently are. Any model you'd want to use on real world population would need to have detection rates >99.9%, even if you're just talking about pre-screening.
- petra 10y ago
  >> it's astonishing how few journalists can do this math. Good journalists usually let 3rd party experts chime in(this news site usually does that), which seems like a useful filter for news quality.
fractalwrench 10y ago
If the method was specifically targeted at high-crime neighbourhoods, the chance of an accurate match would presumably increase. You could also map the entire population of a country and detect previously unknown "criminal" hotspots. Both are awful, authoritarian ideas, but would potentially give useful information to support regular policing. Thankfully, the most likely application of this technology I can see in the near-future is someone making an app that scores your face for criminality.
- tremon 10y ago
  Thankfully, the most likely application of this technology I can see in the near-future is someone making an app that scores your face for criminality. I don't share your positive outlook on law enforcement officials.

aktiur 10y ago

"They take ID photos of 1856 Chinese men [...]. Half of these men were criminals." Because half of the general male population is criminal, of course. The accuracy rate would be very different with a training/testing sample that takes the base rate of criminality into account.

KaiserPro 10y ago

Phrenology is still phrenology even if its digital.

paulajohnson 10y ago

This doesn't "predict criminals", it predicts those who will be convicted of a crime, which is not the same thing. Suppose that people have an unconscious prejudice against those with eyes set close together. They will be disproportionately convicted, and this neural net will find the correlation.

bryanrasmussen 10y ago
Since people assume beautiful people are good what is the correlation between these results and people who are considered ugly in their particular population?

stewhuk 10y ago

I looked at this paper the other day and it looked to me like the non-criminal faces examples were men with shirt collars, whereas the criminal examples were men in t-shirts. If they got above random accuracy, then I wonder if they simply overfitted on that, and the lighting and colour differences between the two styles of photos.

mobiuscog 10y ago

The future of this would make a great film...

YeGoblynQueenne 10y ago

The paper's conclusion claims that " Furthermore, we have discovered that a law of normality for faces of non- criminals. After controlled for race, gender and age, the general law-biding public have facial appearances that vary in a significantly lesser degree than criminals." Given the above I move that the title of this post is changed to "Neural Net trained on mugshots confirms the findings of Phrenology".

spacemanmatt 10y ago

Could it be used to identify those likely to commit election or securities frauds? Or maybe those likely to poison an entire city by ruining their water supply?

stolk 10y ago

Let me guess... the NN was trained to spot tattoos? I bet you tattoos and crime correlate.

kuroguro 10y ago

"In other words, the faces of general law-biding public have a greater degree of resemblance compared with the faces of criminals, or criminals have a higher degree of dissimilarity in facial appearance than normal people" Hmm... so. You look weird -> people treat you worse -> you don't feel like working with them -> higher chance of being a criminal. I know that's a giant leap in reasoning, but that was the first thing that came to mind.