5 ms·

Why would a title mention API/service configuration? This is not common practice, even in academic applied ML papers. They used the default settings, and a tra

Why would a title mention API/service configuration? This is not common practice, even in academic applied ML papers. They used the default settings, and a training set which seems extremely likely to be used in ‘LEA-production’ Any user, including LEAs, is free to use which ever configuration options they like. The criticism with the most statistical implications is the confidence level used. The ACLU were clear they they used the defaults. I can take at face value that moving from 80% to 99% confidence on a sample of 500 faces could produce 0 false positives. However, on the faces of the tens/hundreds of thousands of people that might move through the center of town on any given day, the implications for causing ordinary people serious harm are large. I am personally shocked that Amazon are willing to “recommend” law enforcement actions at any level of confidence, from an unintrospectable machine learning system.

cptskippy 8y ago
The title is akin to saying "We bought <gun maker>'s hunting rifle, pointed it at Congress, pulled the trigger, and it shot 22 members of Congress". It's a tool, you're responsible for how it is used, not the manufacturer. I would imagine the confidence level would be tweaked by Law Enforcement or anyone else based on the results. If you're monitoring cameras for missing persons or shooting suspects and getting 0 hits, you might lower it because it's better to dismiss false positives than the alternative. Conversely you might increase it if you're overwhelmed by the number of false positives.
- d4l3k 8y ago
  Since you're comparing facial recognition to guns and guns are regulated by law and have strict rules for use, perhaps facial recognition should as well? I think there should be a discussion about laws/regulations about facial recognition use. Even if Amazon removes their service there will always be companies that don't care about the ethics as much. Strict rules instead of public shaming seems like the better approach
  - RhodesianHunter 8y ago
    If you screw up with a gun someone dies. If you screw up with facial recognition a human double-checks and nothing happens. Scissors are dangerous but we don't regulate those either.
    - syshum 8y ago
      ummm no... First of it is not a given if you "screw up with a gun someone dies" that is not the only result Further it is laughable to believe that with facial recognition a "human double-checks" we have seen time and time again that new technology entering the realm of criminal justice can be used to wrongfully convict people at an alarming rate. See Hair and Bite mark controversy or the number of Labs that have been compromised in recent years The fact is that a "humans" that are suppose to be "double checking" do not in reality, if the computer spits out "Well the computer says it is person X" that is what the Jury will hear, and the Jury will not question the technology and the person will end up in prison until such time as someone like the Innocent project comes along to invalidate either the technology on the whole or the application of the technology
    - sangnoir 8y ago
      > If you screw up with facial recognition a human double-checks and nothing happens. You have too much faith in humanity: a lot can be, and will be done "because the system says so". Nevermind integrators who will create processes that assume recognition is 100% correct "Yes, I know you're not the wanted criminal, but now that I have tagged you, I have to take you to the station and process you out...Monday morning because that's when Jim is in"
- confounded 8y ago
  > The title is akin to saying "We bought <gun maker>'s hunting rifle, pointed it at Congress, pulled the trigger, and it shot 22 members of Congress". Only if <gun maker> was specifically marketing their hunting rifle to be aimed at members of the public, and for the trigger to be frequently pulled. Happily, gun-makers are far more responsible than that. > It's a tool, you're responsible for how it is used, not the manufacturer Not so. Private businesses can refuse to offer services largely as they wish, and AWS frequently do, based on their terms of service. For example, you are not permitted to run a web-scraper or open mail relay on EC2. You are not permitted to host bestiality on S3. None of these things are illegal. But Amazon have taken the stance that they just don't want them to happen on AWS. I am of the view that mass facial recognition poses more problems for society than running web-scrapers. Many civil liberties organizations have serious concerns about the use of facial recognition for law enforcement, including pro-industry organizations such as the EFF[0]. This is not some new reaction; they have had Know Your Customer guidelines about terms for ML for states since 2011[1]. Even competitors who are incentivized to sell facial recognition tech to LEAs are warning that it's a bad idea for society, with serious and likely lethal implications. Here's the CEO of facial recognition startup Kairos[2]. > ...the confidence level would be tweaked by Law Enforcement This is a large part of the problem. If you don't think so, then we simply have very different visions of the future that we want to live in, and must agree to disagree. [0] https://www.eff.org/wp/law-enforcement-use-face-recognition https://www.eff.org/wp/law-enforcement-use-face-recognition [1] https://www.eff.org/deeplinks/2011/10/it%E2%80%99s-time-know-your-customer-standards-sales-surveillance-equipment https://www.eff.org/deeplinks/2011/10/it%E2%80%99s-time-know... [2] https://techcrunch.com/2018/06/25/facial-recognition-software-is-not-ready-for-use-by-law-enforcement/ https://techcrunch.com/2018/06/25/facial-recognition-softwar...
  - joshgel 8y ago
    > Happily, gun-makers are far more responsible than that. I mean, accurate, but then again they are making guns, so ... low bar i guess.
- MR4D 8y ago
  And when this tool is pointed at a poor black neighborhood, do you think people will question the results just like they did the members of Congress? To me, this is the issue. We make all sorts of statements when it’s congress, but when it’s real life it’s not enough to make the news. After all, how many members of Congress have been shot because a police officer thought their cell phone was a gun? THAT is what the ACLU is trying to point out here.
stevew20 8y ago
I personally believe that facial recognition software will eventually be employed by law enforcement no matter what we do; I also believe that this is a bad thing and lies on the path to a controlling and pervasive police state. However, this article states clearly that there are recommendations to LE on how to use Rekognition in a meaningful and correct manner for LE's use case. I am highly confident that Amazon will provide technical documentation and manuals for a contract of that size; of course Amazon will not be able to conduct oversight on LE operations. LE may missuse it, the way they've missused Stingrays, stoplight cameras, clipper chips, and other devices. You state that having 99% accuracy for identifying an individual will do large scale harm, and I agree; what about the 1 person out of every 100 that is falsly identified? The other side to this coin is the number of people law enforcement would miss recognizing without this software... This argument comes down to human ability vs. machine learning ability, and human discretion in interpreting results. I think with proper use it'll be a benefit to identying culprits quickly. Regardless, it is still pretty evil giving LE any tool that can autonomously monitor people who have not committed any crimes. But who's going to stop them?

cptskippy 8y ago

The title is akin to saying "We bought <gun maker>'s hunting rifle, pointed it at Congress, pulled the trigger, and it shot 22 members of Congress". It's a tool, you're responsible for how it is used, not the manufacturer. I would imagine the confidence level would be tweaked by Law Enforcement or anyone else based on the results. If you're monitoring cameras for missing persons or shooting suspects and getting 0 hits, you might lower it because it's better to dismiss false positives than the alternative. Conversely you might increase it if you're overwhelmed by the number of false positives.

d4l3k 8y ago
Since you're comparing facial recognition to guns and guns are regulated by law and have strict rules for use, perhaps facial recognition should as well? I think there should be a discussion about laws/regulations about facial recognition use. Even if Amazon removes their service there will always be companies that don't care about the ethics as much. Strict rules instead of public shaming seems like the better approach
- RhodesianHunter 8y ago
  If you screw up with a gun someone dies. If you screw up with facial recognition a human double-checks and nothing happens. Scissors are dangerous but we don't regulate those either.
  - syshum 8y ago
    ummm no... First of it is not a given if you "screw up with a gun someone dies" that is not the only result Further it is laughable to believe that with facial recognition a "human double-checks" we have seen time and time again that new technology entering the realm of criminal justice can be used to wrongfully convict people at an alarming rate. See Hair and Bite mark controversy or the number of Labs that have been compromised in recent years The fact is that a "humans" that are suppose to be "double checking" do not in reality, if the computer spits out "Well the computer says it is person X" that is what the Jury will hear, and the Jury will not question the technology and the person will end up in prison until such time as someone like the Innocent project comes along to invalidate either the technology on the whole or the application of the technology
  - sangnoir 8y ago
    > If you screw up with facial recognition a human double-checks and nothing happens. You have too much faith in humanity: a lot can be, and will be done "because the system says so". Nevermind integrators who will create processes that assume recognition is 100% correct "Yes, I know you're not the wanted criminal, but now that I have tagged you, I have to take you to the station and process you out...Monday morning because that's when Jim is in"
confounded 8y ago
> The title is akin to saying "We bought <gun maker>'s hunting rifle, pointed it at Congress, pulled the trigger, and it shot 22 members of Congress". Only if <gun maker> was specifically marketing their hunting rifle to be aimed at members of the public, and for the trigger to be frequently pulled. Happily, gun-makers are far more responsible than that. > It's a tool, you're responsible for how it is used, not the manufacturer Not so. Private businesses can refuse to offer services largely as they wish, and AWS frequently do, based on their terms of service. For example, you are not permitted to run a web-scraper or open mail relay on EC2. You are not permitted to host bestiality on S3. None of these things are illegal. But Amazon have taken the stance that they just don't want them to happen on AWS. I am of the view that mass facial recognition poses more problems for society than running web-scrapers. Many civil liberties organizations have serious concerns about the use of facial recognition for law enforcement, including pro-industry organizations such as the EFF[0]. This is not some new reaction; they have had Know Your Customer guidelines about terms for ML for states since 2011[1]. Even competitors who are incentivized to sell facial recognition tech to LEAs are warning that it's a bad idea for society, with serious and likely lethal implications. Here's the CEO of facial recognition startup Kairos[2]. > ...the confidence level would be tweaked by Law Enforcement This is a large part of the problem. If you don't think so, then we simply have very different visions of the future that we want to live in, and must agree to disagree. [0] https://www.eff.org/wp/law-enforcement-use-face-recognition https://www.eff.org/wp/law-enforcement-use-face-recognition [1] https://www.eff.org/deeplinks/2011/10/it%E2%80%99s-time-know-your-customer-standards-sales-surveillance-equipment https://www.eff.org/deeplinks/2011/10/it%E2%80%99s-time-know... [2] https://techcrunch.com/2018/06/25/facial-recognition-software-is-not-ready-for-use-by-law-enforcement/ https://techcrunch.com/2018/06/25/facial-recognition-softwar...
- joshgel 8y ago
  > Happily, gun-makers are far more responsible than that. I mean, accurate, but then again they are making guns, so ... low bar i guess.
MR4D 8y ago
And when this tool is pointed at a poor black neighborhood, do you think people will question the results just like they did the members of Congress? To me, this is the issue. We make all sorts of statements when it’s congress, but when it’s real life it’s not enough to make the news. After all, how many members of Congress have been shot because a police officer thought their cell phone was a gun? THAT is what the ACLU is trying to point out here.

stevew20 8y ago

I personally believe that facial recognition software will eventually be employed by law enforcement no matter what we do; I also believe that this is a bad thing and lies on the path to a controlling and pervasive police state. However, this article states clearly that there are recommendations to LE on how to use Rekognition in a meaningful and correct manner for LE's use case. I am highly confident that Amazon will provide technical documentation and manuals for a contract of that size; of course Amazon will not be able to conduct oversight on LE operations. LE may missuse it, the way they've missused Stingrays, stoplight cameras, clipper chips, and other devices. You state that having 99% accuracy for identifying an individual will do large scale harm, and I agree; what about the 1 person out of every 100 that is falsly identified? The other side to this coin is the number of people law enforcement would miss recognizing without this software... This argument comes down to human ability vs. machine learning ability, and human discretion in interpreting results. I think with proper use it'll be a benefit to identying culprits quickly. Regardless, it is still pretty evil giving LE any tool that can autonomously monitor people who have not committed any crimes. But who's going to stop them?