19 ms·

I'm a big ACLU supporter, but thought this was poorly done. They never released either the database or the code for this testing, and had configured the recogni

I'm a big ACLU supporter, but thought this was poorly done. They never released either the database or the code for this testing, and had configured the recognition level against Amazon's recommendations.

chishaku 6y ago
> had configured the recognition level against Amazon's recommendations. Citations? My understanding was that the ACLU used the default settings. July 26, 2018 — Amazon states that it guides law enforcement customers to set a threshold of 95% for face recognition. Amazon also notes that, if its face recognition product is used with the default settings, it won’t “identify[] individuals with a reasonable level of certainty.” July 27, 2018 — Amazon writes that even 95% is an unacceptably low threshold, and states that 99% is the appropriate threshold for law enforcement. https://www.aclu.org/press-releases/aclu-comment-new-amazon-statement-responding-face-recognition-technology-test https://www.aclu.org/press-releases/aclu-comment-new-amazon-... Either way, the defaults are the problem if the application is law enforcement. "Defaults have such powerful and pervasive effects on consumer behavior that they could be considered “hidden persuaders” in some settings. Ignoring defaults is not a sound option for marketers or consumer policy makers. The authors identify three theoretical causes of default effects—implied endorsement, cognitive biases, and effort..." https://journals.sagepub.com/doi/10.1509/jppm.10.114 https://journals.sagepub.com/doi/10.1509/jppm.10.114
gregsadetsky 6y ago
In the ACLU's posts, they say that they used the "default match settings". In their response [0] to Amazon's response, the ACLU links to a guide published by Amazon [1] intended to "Identify Persons of Interest for Law Enforcement" that does use `searchFaceRequest?.faceMatchThreshold = 0.85;` (this is still the case today). Fast Company [2] writes about this as well: "The ACLU in both tests used an 80% match confidence threshold, which is Amazon’s default setting, but Amazon says it encourages law enforcement to use a 99% threshold for spotting a match." That bit of the article links to the CompareFaces API documentation [3] which (still) states "By default, only faces with a similarity score of greater than or equal to 80% are returned in the response". Have you seen/read something else about this? [0] https://www.aclu.org/press-releases/aclu-comment-new-amazon-statement-responding-face-recognition-technology-test https://www.aclu.org/press-releases/aclu-comment-new-amazon-... [1] https://aws.amazon.com/blogs/machine-learning/using-amazon-rekognition-to-identify-persons-of-interest-for-law-enforcement/ https://aws.amazon.com/blogs/machine-learning/using-amazon-r... [2] https://www.fastcompany.com/90389905/aclu-amazon-face-recognition-falsely-matched-ca-lawmakers https://www.fastcompany.com/90389905/aclu-amazon-face-recogn... [3] https://docs.aws.amazon.com/rekognition/latest/dg/API_CompareFaces.html https://docs.aws.amazon.com/rekognition/latest/dg/API_Compar...
- tw000001 6y ago
  >Fast Company [2] writes about this as well: "The ACLU in both tests used an 80% match confidence threshold, which is Amazon’s default setting, but Amazon says it encourages law enforcement to use a 99% threshold for spotting a match Then this whole thing is potentially misleading because there's a huge difference between 80% and 99%. It's probably nonlinear and they could possibly see their false matches drop to 0. This is not a fair test - or rather, the conclusions are not quite supported by the parameters. Not that I'm defending police use of facial recognition tech, I think it's abhorrent, though possibly inevitable.
  - gregsadetsky 6y ago
    Just to make sure I understand, which "whole thing" is misleading? The ACLU's test? Amazon's response? As for the test, you say it's not a fair test. The point / conversation right now seems to be about the choice of parameters used by the ACLU. As far as I see / understand, the ACLU used the default parameters (and/or those recommended in the documentation / articles that are still up today with those same non-99% values). What would have been a better / fairer test?
  - rising-sky 6y ago
    What are police departments using? My uninformed guess would be not 99%. I think therein lies the concern...
    - bigiain 6y ago
      My cynical guess would be "whatever the lowest number they can get away with using". I would bet good money that cops KPI goals benefit from false positives, since they'll reward higher "number of identified/interviewed suspects" and "number of arrests" as a positive thing even if "number of convictions" doesn't line up. Even more cynically, I'd bet this is a powerful technique for ambitious cop promotion, and that there's little blowback on fraudulently manipulating parameters that adversely affect POC much more significantly that white people. Thinking about it, I'm now recalling the multiple reports of police departments claiming to not be using clearview.ai, only to have to backtrack when clearview's customer data got popped and it became public knowledge that individual cops were signing up for free trials - which their department/management either chose to hide or didn't know about. That's reasonably compelling circumstantial evidence to me that ambitious cops are quick to jump on unproven and unauthorised technology with insufficient or oversight or with management actively avoiding oversight for them...
JMTQp8lwXL 6y ago
It seems realistic other users would ignore Amazon's recommendations for proper configuration.
- outworlder 6y ago
  Specially given that turning the parameter _down_ will give you more matches. This is great for demos. "No match found" is not so great.
HaloZero 6y ago
I agree. I wish the ACLU would re-run the results at 99%. But Amazon's example post about law enforcement has it set to 85%. I don't think this 99% thing is communicated properly at Amazon if it's getting through blog posts like this. So I think a valid criticism is that we need to make sure that it's higher. https://aws.amazon.com/blogs/machine-learning/using-amazon-rekognition-to-identify-persons-of-interest-for-law-enforcement/ https://aws.amazon.com/blogs/machine-learning/using-amazon-r...
- simion314 6y ago
  Do we know police actually use 99% ?
  - HaloZero 6y ago
    Nope and I doubt they do. But a test of 99% would better for Amazon/ACLU to take on.
    - simion314 6y ago
      I do not understand, a fair test is to replicate reality. Also I am wondering if you have for each city a different software package with it's own config or IT guy that tweaks the config
bko 6y ago
I posted in another comment, but I tried to recreate this using default 70% match. The dataset was 440 images of congressmen and 1,756 mugshots. There were ten mismatches between 70-77% certainty https://medium.com/ml-everything/how-facial-recognition-works-part-4-comparing-congressmen-to-mugshots-bf08b72c3c2a https://medium.com/ml-everything/how-facial-recognition-work...

chishaku 6y ago

> had configured the recognition level against Amazon's recommendations. Citations? My understanding was that the ACLU used the default settings. July 26, 2018 — Amazon states that it guides law enforcement customers to set a threshold of 95% for face recognition. Amazon also notes that, if its face recognition product is used with the default settings, it won’t “identify[] individuals with a reasonable level of certainty.” July 27, 2018 — Amazon writes that even 95% is an unacceptably low threshold, and states that 99% is the appropriate threshold for law enforcement. https://www.aclu.org/press-releases/aclu-comment-new-amazon-statement-responding-face-recognition-technology-test https://www.aclu.org/press-releases/aclu-comment-new-amazon-... Either way, the defaults are the problem if the application is law enforcement. "Defaults have such powerful and pervasive effects on consumer behavior that they could be considered “hidden persuaders” in some settings. Ignoring defaults is not a sound option for marketers or consumer policy makers. The authors identify three theoretical causes of default effects—implied endorsement, cognitive biases, and effort..." https://journals.sagepub.com/doi/10.1509/jppm.10.114 https://journals.sagepub.com/doi/10.1509/jppm.10.114

gregsadetsky 6y ago

In the ACLU's posts, they say that they used the "default match settings". In their response [0] to Amazon's response, the ACLU links to a guide published by Amazon [1] intended to "Identify Persons of Interest for Law Enforcement" that does use `searchFaceRequest?.faceMatchThreshold = 0.85;` (this is still the case today). Fast Company [2] writes about this as well: "The ACLU in both tests used an 80% match confidence threshold, which is Amazon’s default setting, but Amazon says it encourages law enforcement to use a 99% threshold for spotting a match." That bit of the article links to the CompareFaces API documentation [3] which (still) states "By default, only faces with a similarity score of greater than or equal to 80% are returned in the response". Have you seen/read something else about this? [0] https://www.aclu.org/press-releases/aclu-comment-new-amazon-statement-responding-face-recognition-technology-test https://www.aclu.org/press-releases/aclu-comment-new-amazon-... [1] https://aws.amazon.com/blogs/machine-learning/using-amazon-rekognition-to-identify-persons-of-interest-for-law-enforcement/ https://aws.amazon.com/blogs/machine-learning/using-amazon-r... [2] https://www.fastcompany.com/90389905/aclu-amazon-face-recognition-falsely-matched-ca-lawmakers https://www.fastcompany.com/90389905/aclu-amazon-face-recogn... [3] https://docs.aws.amazon.com/rekognition/latest/dg/API_CompareFaces.html https://docs.aws.amazon.com/rekognition/latest/dg/API_Compar...

tw000001 6y ago
>Fast Company [2] writes about this as well: "The ACLU in both tests used an 80% match confidence threshold, which is Amazon’s default setting, but Amazon says it encourages law enforcement to use a 99% threshold for spotting a match Then this whole thing is potentially misleading because there's a huge difference between 80% and 99%. It's probably nonlinear and they could possibly see their false matches drop to 0. This is not a fair test - or rather, the conclusions are not quite supported by the parameters. Not that I'm defending police use of facial recognition tech, I think it's abhorrent, though possibly inevitable.
- gregsadetsky 6y ago
  Just to make sure I understand, which "whole thing" is misleading? The ACLU's test? Amazon's response? As for the test, you say it's not a fair test. The point / conversation right now seems to be about the choice of parameters used by the ACLU. As far as I see / understand, the ACLU used the default parameters (and/or those recommended in the documentation / articles that are still up today with those same non-99% values). What would have been a better / fairer test?
- rising-sky 6y ago
  What are police departments using? My uninformed guess would be not 99%. I think therein lies the concern...
  - bigiain 6y ago
    My cynical guess would be "whatever the lowest number they can get away with using". I would bet good money that cops KPI goals benefit from false positives, since they'll reward higher "number of identified/interviewed suspects" and "number of arrests" as a positive thing even if "number of convictions" doesn't line up. Even more cynically, I'd bet this is a powerful technique for ambitious cop promotion, and that there's little blowback on fraudulently manipulating parameters that adversely affect POC much more significantly that white people. Thinking about it, I'm now recalling the multiple reports of police departments claiming to not be using clearview.ai, only to have to backtrack when clearview's customer data got popped and it became public knowledge that individual cops were signing up for free trials - which their department/management either chose to hide or didn't know about. That's reasonably compelling circumstantial evidence to me that ambitious cops are quick to jump on unproven and unauthorised technology with insufficient or oversight or with management actively avoiding oversight for them...

JMTQp8lwXL 6y ago

It seems realistic other users would ignore Amazon's recommendations for proper configuration.

outworlder 6y ago
Specially given that turning the parameter _down_ will give you more matches. This is great for demos. "No match found" is not so great.

HaloZero 6y ago

I agree. I wish the ACLU would re-run the results at 99%. But Amazon's example post about law enforcement has it set to 85%. I don't think this 99% thing is communicated properly at Amazon if it's getting through blog posts like this. So I think a valid criticism is that we need to make sure that it's higher. https://aws.amazon.com/blogs/machine-learning/using-amazon-rekognition-to-identify-persons-of-interest-for-law-enforcement/ https://aws.amazon.com/blogs/machine-learning/using-amazon-r...

simion314 6y ago
Do we know police actually use 99% ?
- HaloZero 6y ago
  Nope and I doubt they do. But a test of 99% would better for Amazon/ACLU to take on.
  - simion314 6y ago
    I do not understand, a fair test is to replicate reality. Also I am wondering if you have for each city a different software package with it's own config or IT guy that tweaks the config

bko 6y ago

I posted in another comment, but I tried to recreate this using default 70% match. The dataset was 440 images of congressmen and 1,756 mugshots. There were ten mismatches between 70-77% certainty https://medium.com/ml-everything/how-facial-recognition-works-part-4-comparing-congressmen-to-mugshots-bf08b72c3c2a https://medium.com/ml-everything/how-facial-recognition-work...