5 ms·
I'm not only disappointed in the ACLU for misinterpreting the results, but also because I usually have a lot of faith in their competency, and they absolutely s
by komali2 8y ago
I'm not only disappointed in the ACLU for misinterpreting the results, but also because I usually have a lot of faith in their competency, and they absolutely should have seen this response coming from Amazon.
What'd they think Amazon was going to do, roll over and be like "turns our our facial recognition software is racist, woops!"? Now instead of a meaningful dialog, we've got a line in the sand where on one side, the ACLU, champion of the people's rights, doesn't understand technology, and on the other side, facial recognition software is Bad and Evil. Shits so polarizing these days it seems there's no room for negotiation, as much as I'd like there to be.
- obastani 8y agoBut a huge part of the problem is that many users of facial recognition software will not be experts on how to properly use the software. If the ACLU improperly used it, do you really trust law enforcement agencies to do a better job?
- l9k 8y agoThe local cop will not use AWS APIs by himself just like that, connecting it to the surveillance cameras and mugshot databases. Hopefully, and I think this scandal will help, the engineering departments/contractors for the government will develop software that is appropriately configured for each case.
- stochastic_monk 8y agoI don’t think a cop or juror is going to look at the confidence score and accurately understand the reasonable doubt that a classification is incorrect.
- l9k 8y agoThe cops won't just consume the API. Software developed for the specific task will. What I imagine is this: 1. The cop uploads a picture of the suspect taken from camera surveillance at crime scene 2. Software displays the mugshots (and names) of PROBABLE matches from the database with confidence %, color coding according to confidence level, and notice of proper use 3. Cop compares man/woman's picture with what's displayed 4. Investigation continues if necessary
- visarga 8y agoFace recognition is not being touted being as accurate as DNA testing or fingerprints, so why pretend it is anything but a fuzzy ranking by similarity? Many forensic techniques generate probabilistic results.
- confounded 8y agoWhat’s an appropriate configuration for your neighborhood?
- bpchaps 8y agoHah. You clearly haven't worked with any government agencies or their contractors. There will be mistakes, and it won't be just a few. The public will have very little action towards even validating whether they're configured properly. It's gonna be a shit show.
- confounded 8y ago> I'm not only disappointed in the ACLU for misinterpreting the results... Did they? This blog post describes some implementation choices which could make their false positive rate lower. It reveals no information about how Rekognition is being deployed by LEAs, and there’s no meaningful regulation or oversight about how that happens. We can’t tell what the assumptions and implementation choices of local police departments are, because that’s top secret information. Most people acting on the predictions have no idea what they are, let alone the implications. It’s fine to ding the ACLU study on methods —- you can, because they published some. No-one would argue that a toy model based on members of Congress is representative of the public at large. But it’s more representative that a press release from a company that are trying to normalize and monetize mass surveillance via facial recognition.
- l9k 8y agoThe original ACLU title is: Amazon’s Face Recognition Falsely Matched 28 Members of Congress With Mugshots. It was spread on the news with the same conclusion. It puts directly the blame on Amazon and its technology and doesn't mention any configuration they used.
- confounded 8y agoWhy would a title mention API/service configuration? This is not common practice, even in academic applied ML papers. They used the default settings, and a training set which seems extremely likely to be used in ‘LEA-production’ Any user, including LEAs, is free to use which ever configuration options they like. The criticism with the most statistical implications is the confidence level used. The ACLU were clear they they used the defaults. I can take at face value that moving from 80% to 99% confidence on a sample of 500 faces could produce 0 false positives. However, on the faces of the tens/hundreds of thousands of people that might move through the center of town on any given day, the implications for causing ordinary people serious harm are large. I am personally shocked that Amazon are willing to “recommend” law enforcement actions at any level of confidence, from an unintrospectable machine learning system.
- 8y ago
- nerdponx 8y agoAnd yet, if the ACLU never ran their article, we wouldn't be having this conversation right now. I say it was a job well done.
- ryanmonroe 8y agoThe only important part of this conversation is the part that offers a suggestion for the level of confidence you should use when highly accurate matches are important. But we never needed a conversation for that, it's in the documentation (and in the mind of anyone with basic knowledge of statistics, which should include anyone using this statistical tool for anything important).
- ekianjo 8y ago> champion of the people's rights, The former ACLU, yes, but not the current ACLU: they have repeatedly said they dont want to protect Free Speech anymore. https://reason.com/archives/2018/07/26/liberty-makes-us-unfree-says-the-aclu https://reason.com/archives/2018/07/26/liberty-makes-us-unfr...
- jhowell 8y ago> Shits so polarizing these days it seems there's no room for negotiation, as much as I'd like there to be. Far worse if you're being polarized. Also they ACLU hasn't released their methodology, are you jumping to conclusions with the ACLU "doesn't understand technology"?
- sgt101 8y agoI would be on Amazon's side if their terms of service clearly stated "do not use this tool for law enforcement". Do the terms of service state that? If this is used to misidentify someone and then that someone gets shot how will the folks at Amazon feel about that? To me failing to consider that and failing to act appropriately is reckless. It's as reckless as Uber turning off the Lidar and as reckless as Tesla using the road markings as a dominant steering method. When you have to ask "how many innocent dead will it take?" do we not all think that something might just be going wrong!
- TeMPOraL 8y ago> I would be on Amazon's side if their terms of service clearly stated "do not use this tool for law enforcement". Do the terms of service state that? Why on Earth would they put something like that in their ToS? If anything, the place for that is in documentation ("this product may not be suitable for use in law enforcement"), but then again, since when Amazon does have a clue about the specific requirements of government services, to issue blanket statements like that? > It's as reckless as Uber turning off the Lidar and as reckless as Tesla using the road markings as a dominant steering method. Yes, you don't blame the Lidar company for equipping their products with an off switch, nor do you blame whoever designed the road marking detection for Tesla's decision to base their self-driving tech on it.
- sgt101 8y agoI think that this illustrates the challenge of this debate. I believe that your perspective is that Amazon are there to make money, provide a component and then the person using that is responsible for the outcomes. However I think that the person providing the service is responsible for who they sell it to and what for. The analogy that I would draw here is with guns; if you sell a gun to a child you should be accountable for that as a misdeed. Note - that's an analogy to try to convey the point I am making, I'm not comparing the misdeeds.
- komali2 8y agoIn your example though, is Smith and Wesson responsible for the store selling the gun to a child? (In this specific case yes, because they lobby against common sense gun control) For law enforcement, I assume they wouldn't be hitting the Amazon API directly, but rather using software someone sold them which as part of its service uses Amazon's API.