4 ms·

For the record if anyone didn’t know, they ran the software against the state politicians. It misidentified 26 (I think) state politicians as existing criminals

For the record if anyone didn’t know, they ran the software against the state politicians. It misidentified 26 (I think) state politicians as existing criminals. Which I see irony in there.

daenz 7y ago
Being misidentified as a criminal is a feature not a bug.
mc32 7y ago
There are likely more than 26 state politicians who are criminals who have not been discovered yet. It’s gotta be more than 26. The hard part is figuring out which ones are the criminals and which ones are politicians doing their thing.
- dsfyu404ed 7y ago
  Yes, they're probably all authoritarian crooks but that's not the point. The point is they haven't been convicted so they weren't in the data set yet the software thought they were. The false positive rate is unacceptably high.
ChuckNorris89 7y ago
>It misidentified 26 (I think) state politicians as existing criminals. It's both funny and sad at the same time on multiple levels.
briandear 7y ago
Misidentified?
- hasahmed 7y ago
  Tuche
jklontz 7y ago
The AWS Rekognition study I believe you are referring to was fundamentally flawed in two ways. First, the threshold used was the default suggested for one-to-one matching (lower false reject rate, higher false accept rate). It doesn't make sense to use this threshold for search applications, and when the study was reproduced with a more appropriate (higher) threshold there were 0 mis-identifications. Second, law enforcement use of face recognition doesn't even involve the algorithm making a lights-out identification decision. Instead, the most similar faces are presented to the user in ranked order by similarity (like a search engine). It's a tool for generating investigative leads, often preferable to publishing the face image of a wanted perpetrator on local news.
- reaperducer 7y ago
  the threshold used was the default What makes you think that a LEO would change the defaults? They're not computer scientists, they're people who bought a product to perform a specific function. It's a tool for generating investigative leads So were photo books, lie detectors, surveillance cameras, and other tools at first. They became a close enough proxy for "guilt" in the eyes of the users for people to automatically be treated as criminals, even if they'd done nothing wrong.
  - HALtheWise 7y ago
    I believe that there were multiple defaults, and they intentionally chose the wrong one to make a point. Your quote cropping is embarrassing. The analogy would be blaming an anchor manufacturer when the "default anchor" for rowboats doesn't hold an oil tanker.
  - skybrian 7y ago
    The question is how do you prevent abuse without handicapping investigators too much, since presumably you do want cases to be solved if possible?
  - manigandham 7y ago
    Was this the actual product used by law enforcement or just the same software workflow used for a study? Those are entirely different environments and there's no reason to expect the final offering to behave like the academic setup used in a specific report.
dev_dull 7y ago
Sounds like it actually works!
dmitrygr 7y ago
> It misidentified 26 (I think) state politicians as existing criminals. What do you mean "mis"? It correctly identified politicians as criminals. In that, this AI showed itself to be lightyears ahead of the electorate in terms of understanding how politics works.
- bilbo0s 7y ago
  I understand you're just joking, but either way you look at it, the AI failed. Either the politicians are criminals, and this tool was only able to correctly categorize 26 of them. Or the politicians are not criminals, and this tool erroneously categorized 26 of them as such. And that's based only on what we know. The experiment may have given rise in state leaders to more fundamental objections. Consider, what if the 26 misidentified politicians, were largely black or hispanic males? We just need to know more about what's been happening in all the testing to say whether these systems are useful or not. That's all I'm saying.
- Rebelgecko 7y ago
  The politicians didn't have criminal records, but were being matched with the faces of unrelated people that had been convicted of crimes
- GhettoMaestro 7y ago
  Lol absolutely based.

For the record if anyone didn’t know, they ran the software against the state politicians. It misidentified 26 (I think) state politicians as existing criminals. Which I see irony in there.

daenz 7y ago

Being misidentified as a criminal is a feature not a bug.

mc32 7y ago

There are likely more than 26 state politicians who are criminals who have not been discovered yet. It’s gotta be more than 26. The hard part is figuring out which ones are the criminals and which ones are politicians doing their thing.

dsfyu404ed 7y ago
Yes, they're probably all authoritarian crooks but that's not the point. The point is they haven't been convicted so they weren't in the data set yet the software thought they were. The false positive rate is unacceptably high.

ChuckNorris89 7y ago

>It misidentified 26 (I think) state politicians as existing criminals. It's both funny and sad at the same time on multiple levels.

briandear 7y ago

Misidentified?

hasahmed 7y ago
Tuche

jklontz 7y ago

The AWS Rekognition study I believe you are referring to was fundamentally flawed in two ways. First, the threshold used was the default suggested for one-to-one matching (lower false reject rate, higher false accept rate). It doesn't make sense to use this threshold for search applications, and when the study was reproduced with a more appropriate (higher) threshold there were 0 mis-identifications. Second, law enforcement use of face recognition doesn't even involve the algorithm making a lights-out identification decision. Instead, the most similar faces are presented to the user in ranked order by similarity (like a search engine). It's a tool for generating investigative leads, often preferable to publishing the face image of a wanted perpetrator on local news.

reaperducer 7y ago
the threshold used was the default What makes you think that a LEO would change the defaults? They're not computer scientists, they're people who bought a product to perform a specific function. It's a tool for generating investigative leads So were photo books, lie detectors, surveillance cameras, and other tools at first. They became a close enough proxy for "guilt" in the eyes of the users for people to automatically be treated as criminals, even if they'd done nothing wrong.
- HALtheWise 7y ago
  I believe that there were multiple defaults, and they intentionally chose the wrong one to make a point. Your quote cropping is embarrassing. The analogy would be blaming an anchor manufacturer when the "default anchor" for rowboats doesn't hold an oil tanker.
- skybrian 7y ago
  The question is how do you prevent abuse without handicapping investigators too much, since presumably you do want cases to be solved if possible?
- manigandham 7y ago
  Was this the actual product used by law enforcement or just the same software workflow used for a study? Those are entirely different environments and there's no reason to expect the final offering to behave like the academic setup used in a specific report.

dev_dull 7y ago

Sounds like it actually works!

dmitrygr 7y ago

> It misidentified 26 (I think) state politicians as existing criminals. What do you mean "mis"? It correctly identified politicians as criminals. In that, this AI showed itself to be lightyears ahead of the electorate in terms of understanding how politics works.

bilbo0s 7y ago
I understand you're just joking, but either way you look at it, the AI failed. Either the politicians are criminals, and this tool was only able to correctly categorize 26 of them. Or the politicians are not criminals, and this tool erroneously categorized 26 of them as such. And that's based only on what we know. The experiment may have given rise in state leaders to more fundamental objections. Consider, what if the 26 misidentified politicians, were largely black or hispanic males? We just need to know more about what's been happening in all the testing to say whether these systems are useful or not. That's all I'm saying.
Rebelgecko 7y ago
The politicians didn't have criminal records, but were being matched with the faces of unrelated people that had been convicted of crimes
GhettoMaestro 7y ago
Lol absolutely based.