12 ms·

The ACLU is doing a lot of great work to hold government accountable when it comes to facial recognition tech. https://www.aclu.org/press-releases/aclu-challen

The ACLU is doing a lot of great work to hold government accountable when it comes to facial recognition tech. https://www.aclu.org/press-releases/aclu-challenges-fbi-face-recognition-secrecy https://www.aclu.org/press-releases/aclu-challenges-fbi-face... https://www.aclu.org/press-releases/aclu-challenges-dhs-face-recognition-secrecy https://www.aclu.org/press-releases/aclu-challenges-dhs-face... Would be great to see Amazon's support. The ACLU ran an experiment with Rekognition and these are their findings: "Using Rekognition, we built a face database and search tool using 25,000 publicly available arrest photos. Then we searched that database against public photos of every current member of the House and Senate. We used the default match settings that Amazon sets for Rekognition. ... the software incorrectly matched 28 members of Congress, identifying them as other people who have been arrested for a crime. ... Academic research [0] has also already shown that face recognition is less accurate for darker-skinned faces and women. Our results validate this concern: Nearly 40 percent of Rekognition’s false matches in our test were of people of color, even though they make up only 20 percent of Congress." https://www.aclu.org/blog/privacy-technology/surveillance-technologies/amazons-face-recognition-falsely-matched-28 https://www.aclu.org/blog/privacy-technology/surveillance-te... [0]: http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a.pdf http://proceedings.mlr.press/v81/buolamwini18a/buolamwini18a...

chishaku 6y ago
If so inclined, support the ACLU: https://action.aclu.org/give/donate-to-aclu https://action.aclu.org/give/donate-to-aclu
- ve55 6y ago
  The ACLU used to be on the top of my list of my personal favorite charities, but I've found they've become significantly more reactionary in the last few years, prioritizing emotionally outraging causes over utilitarian causes, so don't donate to them any longer. With that said, some of their work is still great, and I'm thankful for it.
  - xrd 6y ago
    I'm scared of that outcome. I'm glad you donated to them in the past so you obviously valued them at some point. Do you have articles to back that up? I'm not trolling; I wonder if this is the message that the "powers-that-be" want us to think. Remember how McDonalds was the victim of a baseless lawsuit? Well, it wasn't actually the case, but that sure benefitted corporations who can now assert most lawsuits against them are frivolous. https://www.caoc.org/?pg=facts https://www.caoc.org/?pg=facts
    - Karunamon 6y ago
      Having read extensively on that case, my opinion changed. I now consider the lawsuit baseless and the verdict primarily the result of a sympathetic plaintiff and an unsympathetic defendant (the stereotypical sweet old lady vs the stereotypical evil money-grubbing mega-corp). Their coffee is just as hot nowadays and to lower the temperature to the degree to where the effect on Stella would have been meaningfully different would result in lukewarm, under-extracted coffee that fewer people would be interested in. Further, the sheer quantity of Mcdonalds coffee moved every year without incident implies user error, rather than product error. Yes, I know what the jury said and how they divided up the blame. I disagree with their conclusion.
      - outworlder 6y ago
        > and to lower the temperature to the degree to where the effect on Stella would have been meaningfully different would result in lukewarm, under-extracted coffee that fewer people would be interested in. Citation needed.
        catalogia 6y ago
        Hot water systems in homes are generally set to not go hotter than 120F, because much hotter than that could burn somebody in seconds. 120F is a bit more than "lukewarm" (which I take to be an exaggeration), but is nevertheless cooler than anybody serves standard coffee at. 140F (60C) water can give you third degree burns in 5 seconds.
hnmullany 6y ago
I'm a big ACLU supporter, but thought this was poorly done. They never released either the database or the code for this testing, and had configured the recognition level against Amazon's recommendations.
- chishaku 6y ago
  > had configured the recognition level against Amazon's recommendations. Citations? My understanding was that the ACLU used the default settings. July 26, 2018 — Amazon states that it guides law enforcement customers to set a threshold of 95% for face recognition. Amazon also notes that, if its face recognition product is used with the default settings, it won’t “identify[] individuals with a reasonable level of certainty.” July 27, 2018 — Amazon writes that even 95% is an unacceptably low threshold, and states that 99% is the appropriate threshold for law enforcement. https://www.aclu.org/press-releases/aclu-comment-new-amazon-statement-responding-face-recognition-technology-test https://www.aclu.org/press-releases/aclu-comment-new-amazon-... Either way, the defaults are the problem if the application is law enforcement. "Defaults have such powerful and pervasive effects on consumer behavior that they could be considered “hidden persuaders” in some settings. Ignoring defaults is not a sound option for marketers or consumer policy makers. The authors identify three theoretical causes of default effects—implied endorsement, cognitive biases, and effort..." https://journals.sagepub.com/doi/10.1509/jppm.10.114 https://journals.sagepub.com/doi/10.1509/jppm.10.114
- gregsadetsky 6y ago
  In the ACLU's posts, they say that they used the "default match settings". In their response [0] to Amazon's response, the ACLU links to a guide published by Amazon [1] intended to "Identify Persons of Interest for Law Enforcement" that does use `searchFaceRequest?.faceMatchThreshold = 0.85;` (this is still the case today). Fast Company [2] writes about this as well: "The ACLU in both tests used an 80% match confidence threshold, which is Amazon’s default setting, but Amazon says it encourages law enforcement to use a 99% threshold for spotting a match." That bit of the article links to the CompareFaces API documentation [3] which (still) states "By default, only faces with a similarity score of greater than or equal to 80% are returned in the response". Have you seen/read something else about this? [0] https://www.aclu.org/press-releases/aclu-comment-new-amazon-statement-responding-face-recognition-technology-test https://www.aclu.org/press-releases/aclu-comment-new-amazon-... [1] https://aws.amazon.com/blogs/machine-learning/using-amazon-rekognition-to-identify-persons-of-interest-for-law-enforcement/ https://aws.amazon.com/blogs/machine-learning/using-amazon-r... [2] https://www.fastcompany.com/90389905/aclu-amazon-face-recognition-falsely-matched-ca-lawmakers https://www.fastcompany.com/90389905/aclu-amazon-face-recogn... [3] https://docs.aws.amazon.com/rekognition/latest/dg/API_CompareFaces.html https://docs.aws.amazon.com/rekognition/latest/dg/API_Compar...
schwarzmx 6y ago
> Would be great to see Amazon's support. From one of the links on the left of the article: https://blog.aboutamazon.com/policy/amazon-donates-10-million-to-organizations-supporting-justice-and-equity https://blog.aboutamazon.com/policy/amazon-donates-10-millio... > Update, June 9: Since announcing our $10 million donation, we’ve heard many employees are making their own contributions—and we’ve decided to match their donations 100% up to $10,000 per employee to these 12 organizations until July 6, 2020.
- chishaku 6y ago
  I was thinking filing amicus briefs in support of the ACLU in the cases mentioned above but this is helpful too.
season2episode3 6y ago
> ... the software incorrectly matched 28 members of Congress, identifying them as other people who have been arrested for a crime. Lol, that's because it's Amazon AI. Do you expect better from the makers of Alexa?
- prepperpotts 6y ago
  It's not like Amazon is bad at AI. The problem they're solving for is legitimately hard, which is why it's so frustrating that it's been put in production in a way that can really harm people.
bko 6y ago
I tried to recreate this using Rekognition. I used a public mugshot database. Here are my results [0] if you're curious about what these false matches look like. [0] https://medium.com/ml-everything/how-facial-recognition-works-part-4-comparing-congressmen-to-mugshots-bf08b72c3c2a https://medium.com/ml-everything/how-facial-recognition-work...
- ed25519FUUU 6y ago
  Some of those false positives are laughably similar. I have no doubt an LEO would be convinced they have the right suspect. I actually think it’s probably better than most people’s naked eye recognition in a crowd.
  - p1necone 6y ago
    Yeah, seems to me like the real problem is that people actually have a lot of doppelgangers, and applying facial recognition to the entire population, even if just as accurate as a real person is just not a useful thing to do.
    - setr 6y ago
      If you had accuracy similar to a human, and assuming this is an existing procedure humans do use, then of course it's useful -- you're doing the same task cheaper/faster. Presumably you'd also use the same techniques as used by humans today to narrow down further, like taking into account location, time, etc of match
      - mumblemumble 6y ago
        > you're doing the same task cheaper/faster Move fast and break lives.
      - couchand 6y ago
        Doing the same thing but faster and cheaper doesn't necessarily mean it's better. If there's a high cost to a procedure, then the user would need to show the value of checking a candidate in some other way for it to be worth the effort. Making it easier and faster means you're more likely to check against random innocent individuals.
deleted 6y ago
[deleted]
sigzero 6y ago
Sounds like Amazon will use the year to improve their poor performing software to me.
cma 6y ago
> Nearly 40 percent of Rekognition’s false matches in our test were of people of color, even though they make up only 20 percent of Congress. What percentage of the arrest photos were people of color? Was it significantly more or less than the 20 percent people of color in congress or about the same?
microdrum 6y ago
"Using Rekognition, we built a face database and search tool using 25,000 publicly available arrest photos. Then we searched that database against public photos of every current member of the House and Senate. We used the default match settings that Amazon sets for Rekognition." Do you understand what a badly designed experiment this is?
- sokoloff 6y ago
  Depending on the outcome you desire, this could be a perfectly designed activity.
raverbashing 6y ago
If I'm not mistaken there's a configurable detection threshold and/or a match score. But most likely people and organizations will think this work like the movies. I wonder what would be the score of the actual faces if we added them to the test set (faces, but not the same photos). Would bigger test sets have photos that match the targets more than themselves?
mtgx 6y ago
PR stunt. At every turn people pleaded with Amazon that police is going to abuse this technology (and shown them they have), Amazon was amazingly tone-deaf talking about how this tech will improve policing, keep society safe, all the regular talking points from pro-state surveillance supporters. Worse, they even went above and beyond with Ring, giving police full access to their data, with no judicial warrants, etc. So yeah, nice try Amazon, but I know you wouldn't have done this if it wasn't looking really bad for you right now, while also attempting to gain some PR points out of it.
AlgorithmicTime 6y ago
The ACLU ignores one of the most important civil liberties, the right to keep and bear arms, and has said they will no longer defend the free speech rights of those they disagree with. They're trash.
a-nikolaev 6y ago
So does it imply that Amazon are not ready yet, so instead of saying "wait wait for one more year (please?)" they try to make a publicity stunt pretending to be good citizens?

chishaku 6y ago

If so inclined, support the ACLU: https://action.aclu.org/give/donate-to-aclu https://action.aclu.org/give/donate-to-aclu

ve55 6y ago
The ACLU used to be on the top of my list of my personal favorite charities, but I've found they've become significantly more reactionary in the last few years, prioritizing emotionally outraging causes over utilitarian causes, so don't donate to them any longer. With that said, some of their work is still great, and I'm thankful for it.
- xrd 6y ago
  I'm scared of that outcome. I'm glad you donated to them in the past so you obviously valued them at some point. Do you have articles to back that up? I'm not trolling; I wonder if this is the message that the "powers-that-be" want us to think. Remember how McDonalds was the victim of a baseless lawsuit? Well, it wasn't actually the case, but that sure benefitted corporations who can now assert most lawsuits against them are frivolous. https://www.caoc.org/?pg=facts https://www.caoc.org/?pg=facts
  - Karunamon 6y ago
    Having read extensively on that case, my opinion changed. I now consider the lawsuit baseless and the verdict primarily the result of a sympathetic plaintiff and an unsympathetic defendant (the stereotypical sweet old lady vs the stereotypical evil money-grubbing mega-corp). Their coffee is just as hot nowadays and to lower the temperature to the degree to where the effect on Stella would have been meaningfully different would result in lukewarm, under-extracted coffee that fewer people would be interested in. Further, the sheer quantity of Mcdonalds coffee moved every year without incident implies user error, rather than product error. Yes, I know what the jury said and how they divided up the blame. I disagree with their conclusion.
    - outworlder 6y ago
      > and to lower the temperature to the degree to where the effect on Stella would have been meaningfully different would result in lukewarm, under-extracted coffee that fewer people would be interested in. Citation needed.
      - catalogia 6y ago
        Hot water systems in homes are generally set to not go hotter than 120F, because much hotter than that could burn somebody in seconds. 120F is a bit more than "lukewarm" (which I take to be an exaggeration), but is nevertheless cooler than anybody serves standard coffee at. 140F (60C) water can give you third degree burns in 5 seconds.

hnmullany 6y ago

I'm a big ACLU supporter, but thought this was poorly done. They never released either the database or the code for this testing, and had configured the recognition level against Amazon's recommendations.

chishaku 6y ago
> had configured the recognition level against Amazon's recommendations. Citations? My understanding was that the ACLU used the default settings. July 26, 2018 — Amazon states that it guides law enforcement customers to set a threshold of 95% for face recognition. Amazon also notes that, if its face recognition product is used with the default settings, it won’t “identify[] individuals with a reasonable level of certainty.” July 27, 2018 — Amazon writes that even 95% is an unacceptably low threshold, and states that 99% is the appropriate threshold for law enforcement. https://www.aclu.org/press-releases/aclu-comment-new-amazon-statement-responding-face-recognition-technology-test https://www.aclu.org/press-releases/aclu-comment-new-amazon-... Either way, the defaults are the problem if the application is law enforcement. "Defaults have such powerful and pervasive effects on consumer behavior that they could be considered “hidden persuaders” in some settings. Ignoring defaults is not a sound option for marketers or consumer policy makers. The authors identify three theoretical causes of default effects—implied endorsement, cognitive biases, and effort..." https://journals.sagepub.com/doi/10.1509/jppm.10.114 https://journals.sagepub.com/doi/10.1509/jppm.10.114
gregsadetsky 6y ago
In the ACLU's posts, they say that they used the "default match settings". In their response [0] to Amazon's response, the ACLU links to a guide published by Amazon [1] intended to "Identify Persons of Interest for Law Enforcement" that does use `searchFaceRequest?.faceMatchThreshold = 0.85;` (this is still the case today). Fast Company [2] writes about this as well: "The ACLU in both tests used an 80% match confidence threshold, which is Amazon’s default setting, but Amazon says it encourages law enforcement to use a 99% threshold for spotting a match." That bit of the article links to the CompareFaces API documentation [3] which (still) states "By default, only faces with a similarity score of greater than or equal to 80% are returned in the response". Have you seen/read something else about this? [0] https://www.aclu.org/press-releases/aclu-comment-new-amazon-statement-responding-face-recognition-technology-test https://www.aclu.org/press-releases/aclu-comment-new-amazon-... [1] https://aws.amazon.com/blogs/machine-learning/using-amazon-rekognition-to-identify-persons-of-interest-for-law-enforcement/ https://aws.amazon.com/blogs/machine-learning/using-amazon-r... [2] https://www.fastcompany.com/90389905/aclu-amazon-face-recognition-falsely-matched-ca-lawmakers https://www.fastcompany.com/90389905/aclu-amazon-face-recogn... [3] https://docs.aws.amazon.com/rekognition/latest/dg/API_CompareFaces.html https://docs.aws.amazon.com/rekognition/latest/dg/API_Compar...

schwarzmx 6y ago

> Would be great to see Amazon's support. From one of the links on the left of the article: https://blog.aboutamazon.com/policy/amazon-donates-10-million-to-organizations-supporting-justice-and-equity https://blog.aboutamazon.com/policy/amazon-donates-10-millio... > Update, June 9: Since announcing our $10 million donation, we’ve heard many employees are making their own contributions—and we’ve decided to match their donations 100% up to $10,000 per employee to these 12 organizations until July 6, 2020.

chishaku 6y ago
I was thinking filing amicus briefs in support of the ACLU in the cases mentioned above but this is helpful too.

season2episode3 6y ago

> ... the software incorrectly matched 28 members of Congress, identifying them as other people who have been arrested for a crime. Lol, that's because it's Amazon AI. Do you expect better from the makers of Alexa?

prepperpotts 6y ago
It's not like Amazon is bad at AI. The problem they're solving for is legitimately hard, which is why it's so frustrating that it's been put in production in a way that can really harm people.

bko 6y ago

I tried to recreate this using Rekognition. I used a public mugshot database. Here are my results [0] if you're curious about what these false matches look like. [0] https://medium.com/ml-everything/how-facial-recognition-works-part-4-comparing-congressmen-to-mugshots-bf08b72c3c2a https://medium.com/ml-everything/how-facial-recognition-work...

ed25519FUUU 6y ago
Some of those false positives are laughably similar. I have no doubt an LEO would be convinced they have the right suspect. I actually think it’s probably better than most people’s naked eye recognition in a crowd.
- p1necone 6y ago
  Yeah, seems to me like the real problem is that people actually have a lot of doppelgangers, and applying facial recognition to the entire population, even if just as accurate as a real person is just not a useful thing to do.
  - setr 6y ago
    If you had accuracy similar to a human, and assuming this is an existing procedure humans do use, then of course it's useful -- you're doing the same task cheaper/faster. Presumably you'd also use the same techniques as used by humans today to narrow down further, like taking into account location, time, etc of match
    - mumblemumble 6y ago
      > you're doing the same task cheaper/faster Move fast and break lives.
    - couchand 6y ago
      Doing the same thing but faster and cheaper doesn't necessarily mean it's better. If there's a high cost to a procedure, then the user would need to show the value of checking a candidate in some other way for it to be worth the effort. Making it easier and faster means you're more likely to check against random innocent individuals.

deleted 6y ago

[deleted]

sigzero 6y ago

Sounds like Amazon will use the year to improve their poor performing software to me.

cma 6y ago

> Nearly 40 percent of Rekognition’s false matches in our test were of people of color, even though they make up only 20 percent of Congress. What percentage of the arrest photos were people of color? Was it significantly more or less than the 20 percent people of color in congress or about the same?

microdrum 6y ago

"Using Rekognition, we built a face database and search tool using 25,000 publicly available arrest photos. Then we searched that database against public photos of every current member of the House and Senate. We used the default match settings that Amazon sets for Rekognition." Do you understand what a badly designed experiment this is?

sokoloff 6y ago
Depending on the outcome you desire, this could be a perfectly designed activity.

raverbashing 6y ago

If I'm not mistaken there's a configurable detection threshold and/or a match score. But most likely people and organizations will think this work like the movies. I wonder what would be the score of the actual faces if we added them to the test set (faces, but not the same photos). Would bigger test sets have photos that match the targets more than themselves?

mtgx 6y ago

PR stunt. At every turn people pleaded with Amazon that police is going to abuse this technology (and shown them they have), Amazon was amazingly tone-deaf talking about how this tech will improve policing, keep society safe, all the regular talking points from pro-state surveillance supporters. Worse, they even went above and beyond with Ring, giving police full access to their data, with no judicial warrants, etc. So yeah, nice try Amazon, but I know you wouldn't have done this if it wasn't looking really bad for you right now, while also attempting to gain some PR points out of it.

AlgorithmicTime 6y ago

The ACLU ignores one of the most important civil liberties, the right to keep and bear arms, and has said they will no longer defend the free speech rights of those they disagree with. They're trash.

a-nikolaev 6y ago

So does it imply that Amazon are not ready yet, so instead of saying "wait wait for one more year (please?)" they try to make a publicity stunt pretending to be good citizens?