5 ms·

TLDR: 1) When the ALCU tested mugshot photos VS congress member photos, the ALCU set it to use an 80% confidence level threshold, which should result in a 5% f

TLDR: 1) When the ALCU tested mugshot photos VS congress member photos, the ALCU set it to use an 80% confidence level threshold, which should result in a 5% false positive rate. There are 535 members of congress, which from this setting should have resulted in 26.75 misidentifications. The ALCU got 28 misidentifications, which is pretty darn close. 2) The ALCU use a comparatively small dataset of photos. Using a different, 30x bigger dataset, and a 99% confidence level resulted in no misidentifications. 3) The police are only supposed to be using this system to narrow down choices, and then have a human sorting out possible matches, which this should do very well. --- My thoughts: The system appears to be functioning exactly according to specifications. The eternal problem of systems is that the specifications do not match what users are actually expecting/needing the system to do. If the ALCU is confused about how it works, the police probably will be as well.

jsty 8y ago
> If the ALCU is confused about how it works, other people, and police, will be to Potentially, although big volume customers like public bodies are almost certainly in contact with eg. AWS solution architects, who would hopefully guide them around pitfalls like this.
- tomnipotent 8y ago
  > who would hopefully guide them around pitfalls like this. The technical staff interfacing with AWS is probably many layers removed from the day-to-day employees that use this sort of software in law enforcement. Federal agencies like the CIA & FBI have trained analysts to do this work, but your local law enforcement agency isn't going to have this level of specialization or training. You can look at the history of any new tool of forensic science and see unfortunate abuse when it's first deployed (like DNA evidence).
  - l9k 8y ago
    I agree with your comment. Now the question is: Is amazon responsible and should it get the bad press? Or should we blame the local cop and his training? Or the police software implementing the technology?
    - tomnipotent 8y ago
      I'm sure there will be plenty of blame to go around. In a car accident, culpability can fall on the driver, the car manufacturer or even the supplier of a faulty part. Regulations will eventually get put into place setting reasonable guidelines for determining negligence and which party (or parties) are at fault. Unfortunately a lot of innocent people will be victims of bad policy and training until then.
- sgt101 8y ago
  Well, this is $12 of work here (see original articles) and one of the attractions of the service is that it is being bought and used at these levels of cost. At scores or $100's architect led support is not possible.
teraflop 8y ago
Nitpick: > the ALCU set it to use an 80% confidence level threshold, which should result in a 5% false positive rate. I'm not sure where the "should" in this sentence is coming from. As far as I know, Amazon's documentation doesn't claim any particular relationship between confidence and false positive rate. The 5% number in the article was simply talking about the actual false positive rate that the ACLU observed. Not to say the conclusions of the article are wrong, though. It certainly seems unreasonable for someone to assume that "80% confidence" would give an FPR of less than 5%.
- danielvf 8y ago
  Thanks!
- shriphani 8y ago
  possibly some internal metric they have? 80% might be the pairwise similarity for a pair of images and across a whole dataset this might have a 5% error rate.
confounded 8y ago
Ah, good thing we’re employing statisticians as beat officers these days! /s The point is that all of these decisions are unregulated, and complicated systems are being put into the hands of people who have very different professional backgrounds (and incentives) to Applied Machine Learning Engineers. - Just because it’s possible to use a better or more representative data set, it doesn’t mean LE agencies can or will - Just because a particular confidence interval is more suitable, doesn’t mean it will be used. What if there’s pressure to hit targets? What if a senior cop sits on the sys admin’s desk and tells the them that “We’ve really gotta nail this guy. He’s armed, dangerous, and on the run”. - What if cops are radioed and told that a man who HQ are 95% sure is the armed killer is in the car/room/office ahead? How will the LEO react and behave when approaching the suspect, based on that information? What if 10k faces from CCTV were put through the system to get that match? What if there are 10 other officers drawing their pistols approaching 95% matches for the same subject at the same time? These are all difficult questions for statisticians and ethicists to answer. We should not be forcing these decisions on LEOs to deal with in real time. And I, personally, would prefer to live in a free country without facial recognition cameras pointing at me. There is no democratic consent for any of this.
- didibus 8y ago
  I agree, but the post is about Amazon being to blame or not. And it seems law enforcement should be regulated in their use of facial recognition, not Amazon. Well, Amazon too, if they used it themselves for wrong doing. But should you regulate the tech itself? Like ban the development of facial recognition all together? Or for identifying human faces? I feel if you regulate, it should be in its proper usage. Which would mean the law enforcement is to blame. Ot maybe you'd need to get an explicit license to be allowed to use facial recognition and Amazon could make sure that only licensed individuals get access to their tech flr example. Point being, nothing here indicates that Reckognition has inherent bias in its design.
balls187 8y ago
> The police are only supposed to be using this system to narrow down choices, and then have a human sorting out possible matches, which this should do very well. I'd rephrase that Amazon recommends (Rekomends?) using a CI of 99% for law enforcement, and that it only be one factor in policing. --- We're getting to a brave new world here, but there is a lot of parallels with the current one. A common trick for reducing speeding tickets is to request things like when the last time the radar was calibrated, and the date of certification of it's operator. Soon, your lawyer can challenge ML based convictions by requesting the coefficients of the model. Also, the police will use every trick in the book to get an arrest, so unless it's clearly illegal (and in some cases that's not enough to stop the police), you can't say the "police should..." do anything.
- sgt101 8y ago
  Where does 99% come from? Who agreed that, why? I think the Amazon folks are shooting from the hip...
  - balls187 8y ago
    I don't know the answers to that. Perhaps the 99% number is why the post was removed (temporarily?). My understanding about Amazon is that they don't have a culture of shooting from the hip. They certainly encourage risk taking, but not without putting together a solid plan with review.

jsty 8y ago

> If the ALCU is confused about how it works, other people, and police, will be to Potentially, although big volume customers like public bodies are almost certainly in contact with eg. AWS solution architects, who would hopefully guide them around pitfalls like this.

tomnipotent 8y ago
> who would hopefully guide them around pitfalls like this. The technical staff interfacing with AWS is probably many layers removed from the day-to-day employees that use this sort of software in law enforcement. Federal agencies like the CIA & FBI have trained analysts to do this work, but your local law enforcement agency isn't going to have this level of specialization or training. You can look at the history of any new tool of forensic science and see unfortunate abuse when it's first deployed (like DNA evidence).
- l9k 8y ago
  I agree with your comment. Now the question is: Is amazon responsible and should it get the bad press? Or should we blame the local cop and his training? Or the police software implementing the technology?
  - tomnipotent 8y ago
    I'm sure there will be plenty of blame to go around. In a car accident, culpability can fall on the driver, the car manufacturer or even the supplier of a faulty part. Regulations will eventually get put into place setting reasonable guidelines for determining negligence and which party (or parties) are at fault. Unfortunately a lot of innocent people will be victims of bad policy and training until then.
sgt101 8y ago
Well, this is $12 of work here (see original articles) and one of the attractions of the service is that it is being bought and used at these levels of cost. At scores or $100's architect led support is not possible.

teraflop 8y ago

Nitpick: > the ALCU set it to use an 80% confidence level threshold, which should result in a 5% false positive rate. I'm not sure where the "should" in this sentence is coming from. As far as I know, Amazon's documentation doesn't claim any particular relationship between confidence and false positive rate. The 5% number in the article was simply talking about the actual false positive rate that the ACLU observed. Not to say the conclusions of the article are wrong, though. It certainly seems unreasonable for someone to assume that "80% confidence" would give an FPR of less than 5%.

danielvf 8y ago
Thanks!
shriphani 8y ago
possibly some internal metric they have? 80% might be the pairwise similarity for a pair of images and across a whole dataset this might have a 5% error rate.

confounded 8y ago

Ah, good thing we’re employing statisticians as beat officers these days! /s The point is that all of these decisions are unregulated, and complicated systems are being put into the hands of people who have very different professional backgrounds (and incentives) to Applied Machine Learning Engineers. - Just because it’s possible to use a better or more representative data set, it doesn’t mean LE agencies can or will - Just because a particular confidence interval is more suitable, doesn’t mean it will be used. What if there’s pressure to hit targets? What if a senior cop sits on the sys admin’s desk and tells the them that “We’ve really gotta nail this guy. He’s armed, dangerous, and on the run”. - What if cops are radioed and told that a man who HQ are 95% sure is the armed killer is in the car/room/office ahead? How will the LEO react and behave when approaching the suspect, based on that information? What if 10k faces from CCTV were put through the system to get that match? What if there are 10 other officers drawing their pistols approaching 95% matches for the same subject at the same time? These are all difficult questions for statisticians and ethicists to answer. We should not be forcing these decisions on LEOs to deal with in real time. And I, personally, would prefer to live in a free country without facial recognition cameras pointing at me. There is no democratic consent for any of this.

didibus 8y ago
I agree, but the post is about Amazon being to blame or not. And it seems law enforcement should be regulated in their use of facial recognition, not Amazon. Well, Amazon too, if they used it themselves for wrong doing. But should you regulate the tech itself? Like ban the development of facial recognition all together? Or for identifying human faces? I feel if you regulate, it should be in its proper usage. Which would mean the law enforcement is to blame. Ot maybe you'd need to get an explicit license to be allowed to use facial recognition and Amazon could make sure that only licensed individuals get access to their tech flr example. Point being, nothing here indicates that Reckognition has inherent bias in its design.

balls187 8y ago

> The police are only supposed to be using this system to narrow down choices, and then have a human sorting out possible matches, which this should do very well. I'd rephrase that Amazon recommends (Rekomends?) using a CI of 99% for law enforcement, and that it only be one factor in policing. --- We're getting to a brave new world here, but there is a lot of parallels with the current one. A common trick for reducing speeding tickets is to request things like when the last time the radar was calibrated, and the date of certification of it's operator. Soon, your lawyer can challenge ML based convictions by requesting the coefficients of the model. Also, the police will use every trick in the book to get an arrest, so unless it's clearly illegal (and in some cases that's not enough to stop the police), you can't say the "police should..." do anything.

sgt101 8y ago
Where does 99% come from? Who agreed that, why? I think the Amazon folks are shooting from the hip...
- balls187 8y ago
  I don't know the answers to that. Perhaps the 99% number is why the post was removed (temporarily?). My understanding about Amazon is that they don't have a culture of shooting from the hip. They certainly encourage risk taking, but not without putting together a solid plan with review.