9 ms·

Amazon’s Face Recognition Falsely Matched 28 Members of Congress with Mugshots

228 points by okket 8y ago

nolite 8y ago
predictive analytics..?
- rebuilder 8y ago
  So phrenology again? IIRC, that idea didn't exactly hold up to scrutiny.
  - deleted 8y ago
    [deleted]
  - onemoresoop 8y ago
    I think it was humor and made me laugh a bit.. phrenology cannot be taken seriously
mnx 8y ago
I don't see any mention of them verifying those members of congress are not in fact the same people as the mugshots (/s)
- yjftsjthsd-h 8y ago
  I admit that was my first interpretation before I remembered the issues with accuracy.
ggambetta 8y ago
Maybe their face recognition tech can see the future?
- hanselot 8y ago
  inb4 downvoted for racism or satanism or some shit.
- sheeshkebab 8y ago
  Or past
hanselot 8y ago
What percentage of people in the openly available mugshot database were people of colour, and why should that not be relevant?
- bunderbunder 8y ago
  It's relevant, but not as an excuse to dismiss the problem. One of the perennial problems with machine learning is that it has a tendency to intensify pre-existing biases in the system. It's not just that the algorithm tends to reflect biases in the source data, it's that that reflection tends to encourage the people using the system, who generally have some role in creating that source data in the first place, to become even more biased.
  - hanselot 8y ago
    biasception. This can be the title of a new film where ai-powered echo chambers produce progressively more polarized societies until civil war and anarchy destroy all of humankind. Oh wait, this might just be reality already.
    - bunderbunder 8y ago
      Society's honestly not becoming particularly polarized, on anything remotely like a historical scale. To take US history, for example, I can think of several times in the past century alone that the public sphere was more acrimonious. I haven't made much of a study of primary sources, but I wouldn't be surprised if a common thing that happened every one of those times was that people would start trying to figure out how to turn "appeal to acrimony" into a convincing debate tactic.
    - dang 8y ago
      We've banned this account for repeatedly breaking the site guidelines. If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
joemaller1 8y ago
Things I want to know: - Show us the side-by-side images of the false-positives. Are the matches plausible? - What is the demographic distribution of the mugshot database? If the data is disproportionately biased, then that bias would be reflected in the false-positives. A casual skimming of some mughot websites shows a potentially significant racial bias.
- hanselot 8y ago
  How dare you question our great overlords the ACLU. They would never use statistics to find results favourable to their hypothesis.
  - dang 8y ago
    Maybe not, but please don't post unsubstantive comments here.
- jerf 8y ago
  I have a ton of quibbles with the article and methodology, honestly. I suspect it's just straight-up so flawed as to be irrelevant to the discussion. At the same time, they do still have the correct final conclusion. Dragnet facial recognition is a bad idea that will produce too many false positives to be of use. Or at the very least, dragnet facial recognition used by people who put too much faith in it is a bad idea. It's at most useful to produce flags for slightly more attention; on its own, I would not consider it anywhere close to enough to arrest someone on. And it can serve as a demonstration of how a clumsily-put-together system can be used to produce bad results, so, if we correctly assume that there may be people bodging together a system in much the same way the ACLU did, well, hey, it's a valid result then!
  - Kalium 8y ago
    I really want to see their methodology. I'd love to replicate it and play with certainty levels to see what happens.
- mrweasel 8y ago
aarong11 8y ago
Could this be down to the people training the algorithm being predominantly middle class white people? It's a pretty well known phenomenon that people are bad at recognising features of people of other races.
- rcar 8y ago
  Racial minorities are gong to be greatly overrepresented in a collection of mugshots, so it's almost certainly just the case that it's easier to make mistakes in that direction. If the algorithm is, say, 99.99% accurate on faces that are the same race, gender, and roughly age, and there's 100 faces in there that match those, it'll correctly say no match .9999^100 = 99% of the time. If there's 5000, it's only going to find no match .9999^5000 = 60% of the time.
- pandasun 8y ago
  >middle class ? That doesn't add up to the rest of your comment.
sigzero 8y ago
Are you sure? lol
lbriner 8y ago
The scary part is that people believe "science" because it is created by people who they think are "clever" and thefore not likely to be wrong. How many people have been falsely convicted because "DNA"? 1B-1 odds of a match, "and sir, yet you claim you were not even in the area?". There should be a way of recognising the parts of the science that are basically correct and the parts that are either less reliable, open to bias or could simply be broken due to incorrect process/mistake in the lab.
- ceejayoz 8y ago
  This is a long, horrific read about exactly that, resulting in the execution of an almost certainly innocent man. https://www.newyorker.com/magazine/2009/09/07/trial-by-fire https://www.newyorker.com/magazine/2009/09/07/trial-by-fire > Many arson investigators, it turned out, had only a high-school education. In most states, in order to be certified, investigators had to take a forty-hour course on fire investigation, and pass a written exam. Often, the bulk of an investigator’s training came on the job, learning from “old-timers” in the field, who passed down a body of wisdom about the telltale signs of arson, even though a study in 1977 warned that there was nothing in “the scientific literature to substantiate their validity.” > In 1992, the National Fire Protection Association, which promotes fire prevention and safety, published its first scientifically based guidelines to arson investigation. Still, many arson investigators believed that what they did was more an art than a science—a blend of experience and intuition. In 1997, the International Association of Arson Investigators filed a legal brief arguing that arson sleuths should not be bound by a 1993 Supreme Court decision requiring experts who testified at trials to adhere to the scientific method. What arson sleuths did, the brief claimed, was “less scientific.” By 2000, after the courts had rejected such claims, arson investigators increasingly recognized the scientific method, but there remained great variance in the field, with many practitioners still relying on the unverified techniques that had been used for generations. “People investigated fire largely with a flat-earth approach,” Hurst told me. “It looks like arson—therefore, it’s arson.” He went on, “My view is you have to have a scientific basis. Otherwise, it’s no different than witch-hunting.”
cthalupa 8y ago
https://www.theverge.com/2018/7/26/17615634/amazon-rekognition-aclu-mug-shot-congress-facial-recognition https://www.theverge.com/2018/7/26/17615634/amazon-rekogniti... >Reached by The Verge, an Amazon spokesperson attributed the results to poor calibration. The ACLU’s tests were performed using Rekognition’s default confidence threshold of 80 percent — but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. Presented without real comment on my part.
- lawlessone 8y ago
  Would that mean 5% could be false positives? That would be a lot.
  - Kalium 8y ago
    From the AWS docs: > A confidence score is a number between 0 and 100 that indicates the probability that a given prediction is correct. In the tropical beach example, if the object and scene detection process returns a confidence score of 99 for the label ‘Water’ and 35 for the label ‘Palm Tree’, then it is more likely that the image contains water but not a palm tree. > Applications that are very sensitive to detection errors (false positives) should discard results associated with confidence scores below a certain threshold. The optimum threshold depends on the application. In many cases, you will get the best user experience by setting minimum confidence values higher than the default value. Which I am interpreting as similar to your question, but perhaps should not be understood as a statistical probability.
- raverbashing 8y ago
  Thinking of an Onion headline: "This tech sucks" says person trying to use niche technology for the first time without knowing its specificities and limitations.
  - adamson 8y ago
    I guess the question is: do we trust any operator to know the specificities and limitations? I for one wouldn't trust Joe Cop to use something like this with the degree of discretion or understanding of statistical nuance called for here.
sadris 8y ago
Internet used to be delivered through unreliable 28.8kbps modems. I didn't think for a second that we should cease using the Internet. Bone marrow transplants for cancer treatment used to have a 20% failure rate. I didn't think for a second that we should cease using bone marrow transplants. And yet why does the ACLU think we should cease using a technology to deliver potential location hits on wanted criminals because its not 100% perfect?
- mrweasel 8y ago
  The problem is that it's only 95% perfect in this case (28 out of 534, being identified as criminals). That a pretty big margin of error when the result is possible arrest. The scope of its use is also in question, sure if you looking a someone who kidnapped a child, 95% is good enough. If you use it for any minor violation you risk harassing a large percentage of the population, who then will need to prove that it wasn't them. Some will flat out deny being the guilty party, and there's a 5% margin of error, so they may very well be right. Then what will you do, drop the charge? In that case what's the point. Or will you spend police time finding evidence that some random person stole $10 worth of good at the super market? The cost associated with being just 5% wrong is huge, and that's not including the emotional damage done to falsely accused citizens, or the decrease level of trust in law enforcement.
  - sadris 8y ago
    They weren't "identified as criminals." They were identified as possibly being criminals. (And as other posters have shown, they were identified with only 80% confidence as being a criminal.) And an officer can review the evidence presented by the report and make a human determination to follow up with a physical arrest.
    - SmellyGeekBoy 8y ago
      Perhaps more of an issue in a country where being arrested can involve being murdered by the police.
    - akira2501 8y ago
AdmiralAsshat 8y ago
I don't see why this is a bad thing. Falsely arrest 28 members of Congress due to poor face recognition, and the problem of facial recognition in law enforcement is resolved the next day.
- deleted 8y ago
  [deleted]
- dragontamer 8y ago
  Congress doesn't stick up for each other. As soon as those 28-Congressmen are gone, they'll start passing legislation that those 28-people didn't want. Congress is a battleground. Not a club or secret society. You send representitives from your state to try and get a slice of the pie, and I send my reps from my area to try and get a slice for me.
twothamendment 8y ago
They have Montana Rep. Greg Gianforte in the list of false positives. Did they call it a false positive because they are sure he wasn't included in the 25,000 mugs they loaded? His mug is out there. https://www.google.com/amp/s/amp.usatoday.com/amp/756343001 https://www.google.com/amp/s/amp.usatoday.com/amp/756343001
jamisteven 8y ago
In other news, Amazons "Freudian Slip" Machine is 100% accurate.
deleted 8y ago
[deleted]
aurizon 8y ago
Only 28? Should be a complete 1:1, meaning they all seem crooked in one way or another...
alphabettsy 8y ago
Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those are real considerations for any program that has the potential to help ruin someone’s life. It also possibly highlights the fact that algorithms are not immune to bias when they are designed by humans. Obviously I don’t think this bias is intentional, but so much of it isn’t and happens anyway.
- dfundako 8y ago
  I think a fun thought exercise is finding the fine line between tech and guns/alcohol/cars. You cannot sue a gun/car/alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can be used improperly and have disastrous results, like a car or gun), could Amazon or a tech administering it be liable for someone falsely imprisoned?
  - vorpalhex 8y ago
    > could Amazon or a tech administering it be liable for someone falsely imprisoned? I would wager "absolutely not" for the exact same issue as firearms or cars. The person who mis-applied the technology or a middle man vendor though? Unless they fall under qualified immunity, there's your fall guy.
  - guitarbill 8y ago
    It's still ethically questionable. In fact I'm struggling to come up with a better example than facial recognition tech (except other mass surveillance). Maybe cutting corners while developing driverless cars that results in the death of a pedestrian. Almost every engineering discipline has a code of ethics [0][1][2][3]. It's time software "engineering" grew up and did the same. I rarely see ethics mentioned on HN, and granted, people's view differ. But it's weird we're not having that conversation at all. [0] https://www.raeng.org.uk/policy/engineering-ethics/ethics https://www.raeng.org.uk/policy/engineering-ethics/ethics [1] https://www.ieee.org/about/corporate/governance/p7-8.html https://www.ieee.org/about/corporate/governance/p7-8.html [2] https://www.nspe.org/resources/ethics/code-ethics https://www.nspe.org/resources/ethics/code-ethics [3] http://www.asce.org/code-of-ethics/ http://www.asce.org/code-of-ethics/
ejlangev 8y ago
Amazon really has to take this back to the drawing board. ~5% accuracy is pretty terrible...
mc32 8y ago
There are definitely concerns about false positives, but it has to be compared to the current system. Are the results, effectiveness, better? When the gov steers opinion, we call it manufactured consent, when public advocacy organizations engage in sloppy methodology to further a cause, I propose calling it manufactured outrage.
drpgq 8y ago
I work in the industry and obviously I can't say for certain, but I don't think Amazon has anywhere near the most accurate face recognition software available. If you want to see where the tech is in terms of performance, the various NIST tests are highly informative.
exabrial 8y ago
Are they sure this was a mistake? I wouldn't be terribly surprised to find out if that many had mugshots.
reaperducer 8y ago
Well, it's not like police are using Amazon's face recognition technology in dragnets. Oh, wait: https://www.usatoday.com/story/tech/talkingtech/2018/06/29/capital-gazette-gunman-identified-using-facial-recognition-technology/744344002/ https://www.usatoday.com/story/tech/talkingtech/2018/06/29/c...
ericcumbee 8y ago
At the risk of being down voted....it recognizes 28 members of congress as criminals....i’d Call that a good start.
squozzer 8y ago
America is in too much danger to take any chances. The 28 positives should be held indefinitely until they can prove they're innocent. Presumption of Guilt is the new normal, isn't it?
andrewguenther 8y ago
> People of color were disproportionately falsely matched in our test. They are trying to make this racially charged without giving enough information to verify their claims. If you use a dataset of mugshots, that's statistically going to have more data on people of color. If you have more data on people of color, it is more likely to match people of color. Claiming the algorithm is racist because your data is racist is inflammatory bullshit.
- spamizbad 8y ago
  How is stating a fact- it disproportionately matched people of color- somehow the same as claiming racism?
  - hoschicz 8y ago
    The fact is irrelevant to the problem and there is no reason to mention it, if they want the debate to stay professional and not emotional. The whole racism angle of the article seems way too loaded for me.
- rrego 8y ago
  Pretty lousy argument. Firstly, the majority of inmates in the US are white (58%). I'd assume mug shot stats are similar, or at the very least not mostly black people. Moreover, if a algorithm enforces bias to the detriment of inoccents, it's a bad algorithm
  - andrewguenther 8y ago
    > Firstly, the majority of inmates in the US are white (58%). I'd assume mug shot stats are similar, or at the very least not mostly black people. I actually did not know this. Thank you. I had conflated incarceration rate with inmate population. If anyone is curious for a source, see here: https://www.bop.gov/about/statistics/statistics_inmate_race.jsp https://www.bop.gov/about/statistics/statistics_inmate_race.... > Moreover, if a algorithm enforces bias to the detriment of inoccents, it's a bad algorithm I agree with this 100%. But they haven't provided enough information about their dataset to make this conclusion. If they provided enough information for someone to independently analyze the data and reproduce the experiment, I would flip immediately.
j_lane 8y ago
maybe we should let the cops have this...
stretchwithme 8y ago
People have similar faces. Who knew? The technology is not perfect. And should never be used as evidence of a crime, just as an indication that two images might be of the same person and to have humans look at them. And then those humans will also make mistakes. People do look alike. And there are twins. I think a jury should require more evidence than just similar appearance. But such a match is a strong indication of where to look.
- see-dante 8y ago
  I don't think that's the point -- the problem isn't that "oh there are other processes that [...]," it's that this wave of technology is misused, is bias, and care must be taken when any of the signals we create. This is especially important as automation enters the equation.
knorker 8y ago
Sure. "Falsely" :-)
imnotlost 8y ago
The algo missed 507 of those crooks!

228 points by okket 8y ago

nolite 8y ago

predictive analytics..?

rebuilder 8y ago
So phrenology again? IIRC, that idea didn't exactly hold up to scrutiny.
- deleted 8y ago
  [deleted]
- onemoresoop 8y ago
  I think it was humor and made me laugh a bit.. phrenology cannot be taken seriously

mnx 8y ago

I don't see any mention of them verifying those members of congress are not in fact the same people as the mugshots (/s)

yjftsjthsd-h 8y ago
I admit that was my first interpretation before I remembered the issues with accuracy.

ggambetta 8y ago

Maybe their face recognition tech can see the future?

hanselot 8y ago
inb4 downvoted for racism or satanism or some shit.
sheeshkebab 8y ago
Or past

hanselot 8y ago

What percentage of people in the openly available mugshot database were people of colour, and why should that not be relevant?

bunderbunder 8y ago
It's relevant, but not as an excuse to dismiss the problem. One of the perennial problems with machine learning is that it has a tendency to intensify pre-existing biases in the system. It's not just that the algorithm tends to reflect biases in the source data, it's that that reflection tends to encourage the people using the system, who generally have some role in creating that source data in the first place, to become even more biased.
- hanselot 8y ago
  biasception. This can be the title of a new film where ai-powered echo chambers produce progressively more polarized societies until civil war and anarchy destroy all of humankind. Oh wait, this might just be reality already.
  - bunderbunder 8y ago
    Society's honestly not becoming particularly polarized, on anything remotely like a historical scale. To take US history, for example, I can think of several times in the past century alone that the public sphere was more acrimonious. I haven't made much of a study of primary sources, but I wouldn't be surprised if a common thing that happened every one of those times was that people would start trying to figure out how to turn "appeal to acrimony" into a convincing debate tactic.
  - dang 8y ago
    We've banned this account for repeatedly breaking the site guidelines. If you don't want to be banned, you're welcome to email hn@ycombinator.com and give us reason to believe that you'll follow the rules in the future. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

joemaller1 8y ago

Things I want to know: - Show us the side-by-side images of the false-positives. Are the matches plausible? - What is the demographic distribution of the mugshot database? If the data is disproportionately biased, then that bias would be reflected in the false-positives. A casual skimming of some mughot websites shows a potentially significant racial bias.

hanselot 8y ago
How dare you question our great overlords the ACLU. They would never use statistics to find results favourable to their hypothesis.
- dang 8y ago
  Maybe not, but please don't post unsubstantive comments here.
jerf 8y ago
I have a ton of quibbles with the article and methodology, honestly. I suspect it's just straight-up so flawed as to be irrelevant to the discussion. At the same time, they do still have the correct final conclusion. Dragnet facial recognition is a bad idea that will produce too many false positives to be of use. Or at the very least, dragnet facial recognition used by people who put too much faith in it is a bad idea. It's at most useful to produce flags for slightly more attention; on its own, I would not consider it anywhere close to enough to arrest someone on. And it can serve as a demonstration of how a clumsily-put-together system can be used to produce bad results, so, if we correctly assume that there may be people bodging together a system in much the same way the ACLU did, well, hey, it's a valid result then!
- Kalium 8y ago
  I really want to see their methodology. I'd love to replicate it and play with certainty levels to see what happens.
mrweasel 8y ago

aarong11 8y ago

Could this be down to the people training the algorithm being predominantly middle class white people? It's a pretty well known phenomenon that people are bad at recognising features of people of other races.

rcar 8y ago
Racial minorities are gong to be greatly overrepresented in a collection of mugshots, so it's almost certainly just the case that it's easier to make mistakes in that direction. If the algorithm is, say, 99.99% accurate on faces that are the same race, gender, and roughly age, and there's 100 faces in there that match those, it'll correctly say no match .9999^100 = 99% of the time. If there's 5000, it's only going to find no match .9999^5000 = 60% of the time.
pandasun 8y ago
>middle class ? That doesn't add up to the rest of your comment.

sigzero 8y ago

Are you sure? lol

lbriner 8y ago

The scary part is that people believe "science" because it is created by people who they think are "clever" and thefore not likely to be wrong. How many people have been falsely convicted because "DNA"? 1B-1 odds of a match, "and sir, yet you claim you were not even in the area?". There should be a way of recognising the parts of the science that are basically correct and the parts that are either less reliable, open to bias or could simply be broken due to incorrect process/mistake in the lab.

ceejayoz 8y ago
This is a long, horrific read about exactly that, resulting in the execution of an almost certainly innocent man. https://www.newyorker.com/magazine/2009/09/07/trial-by-fire https://www.newyorker.com/magazine/2009/09/07/trial-by-fire > Many arson investigators, it turned out, had only a high-school education. In most states, in order to be certified, investigators had to take a forty-hour course on fire investigation, and pass a written exam. Often, the bulk of an investigator’s training came on the job, learning from “old-timers” in the field, who passed down a body of wisdom about the telltale signs of arson, even though a study in 1977 warned that there was nothing in “the scientific literature to substantiate their validity.” > In 1992, the National Fire Protection Association, which promotes fire prevention and safety, published its first scientifically based guidelines to arson investigation. Still, many arson investigators believed that what they did was more an art than a science—a blend of experience and intuition. In 1997, the International Association of Arson Investigators filed a legal brief arguing that arson sleuths should not be bound by a 1993 Supreme Court decision requiring experts who testified at trials to adhere to the scientific method. What arson sleuths did, the brief claimed, was “less scientific.” By 2000, after the courts had rejected such claims, arson investigators increasingly recognized the scientific method, but there remained great variance in the field, with many practitioners still relying on the unverified techniques that had been used for generations. “People investigated fire largely with a flat-earth approach,” Hurst told me. “It looks like arson—therefore, it’s arson.” He went on, “My view is you have to have a scientific basis. Otherwise, it’s no different than witch-hunting.”

cthalupa 8y ago

https://www.theverge.com/2018/7/26/17615634/amazon-rekognition-aclu-mug-shot-congress-facial-recognition https://www.theverge.com/2018/7/26/17615634/amazon-rekogniti... >Reached by The Verge, an Amazon spokesperson attributed the results to poor calibration. The ACLU’s tests were performed using Rekognition’s default confidence threshold of 80 percent — but Amazon says it recommends at least a 95 percent threshold for law enforcement applications where a false ID might have more significant consequences. Presented without real comment on my part.

lawlessone 8y ago
Would that mean 5% could be false positives? That would be a lot.
- Kalium 8y ago
  From the AWS docs: > A confidence score is a number between 0 and 100 that indicates the probability that a given prediction is correct. In the tropical beach example, if the object and scene detection process returns a confidence score of 99 for the label ‘Water’ and 35 for the label ‘Palm Tree’, then it is more likely that the image contains water but not a palm tree. > Applications that are very sensitive to detection errors (false positives) should discard results associated with confidence scores below a certain threshold. The optimum threshold depends on the application. In many cases, you will get the best user experience by setting minimum confidence values higher than the default value. Which I am interpreting as similar to your question, but perhaps should not be understood as a statistical probability.
raverbashing 8y ago
Thinking of an Onion headline: "This tech sucks" says person trying to use niche technology for the first time without knowing its specificities and limitations.
- adamson 8y ago
  I guess the question is: do we trust any operator to know the specificities and limitations? I for one wouldn't trust Joe Cop to use something like this with the degree of discretion or understanding of statistical nuance called for here.

sadris 8y ago

Internet used to be delivered through unreliable 28.8kbps modems. I didn't think for a second that we should cease using the Internet. Bone marrow transplants for cancer treatment used to have a 20% failure rate. I didn't think for a second that we should cease using bone marrow transplants. And yet why does the ACLU think we should cease using a technology to deliver potential location hits on wanted criminals because its not 100% perfect?

mrweasel 8y ago
The problem is that it's only 95% perfect in this case (28 out of 534, being identified as criminals). That a pretty big margin of error when the result is possible arrest. The scope of its use is also in question, sure if you looking a someone who kidnapped a child, 95% is good enough. If you use it for any minor violation you risk harassing a large percentage of the population, who then will need to prove that it wasn't them. Some will flat out deny being the guilty party, and there's a 5% margin of error, so they may very well be right. Then what will you do, drop the charge? In that case what's the point. Or will you spend police time finding evidence that some random person stole $10 worth of good at the super market? The cost associated with being just 5% wrong is huge, and that's not including the emotional damage done to falsely accused citizens, or the decrease level of trust in law enforcement.
- sadris 8y ago
  They weren't "identified as criminals." They were identified as possibly being criminals. (And as other posters have shown, they were identified with only 80% confidence as being a criminal.) And an officer can review the evidence presented by the report and make a human determination to follow up with a physical arrest.
  - SmellyGeekBoy 8y ago
    Perhaps more of an issue in a country where being arrested can involve being murdered by the police.
  - akira2501 8y ago

AdmiralAsshat 8y ago

I don't see why this is a bad thing. Falsely arrest 28 members of Congress due to poor face recognition, and the problem of facial recognition in law enforcement is resolved the next day.

deleted 8y ago
[deleted]
dragontamer 8y ago
Congress doesn't stick up for each other. As soon as those 28-Congressmen are gone, they'll start passing legislation that those 28-people didn't want. Congress is a battleground. Not a club or secret society. You send representitives from your state to try and get a slice of the pie, and I send my reps from my area to try and get a slice for me.

twothamendment 8y ago

They have Montana Rep. Greg Gianforte in the list of false positives. Did they call it a false positive because they are sure he wasn't included in the 25,000 mugs they loaded? His mug is out there. https://www.google.com/amp/s/amp.usatoday.com/amp/756343001 https://www.google.com/amp/s/amp.usatoday.com/amp/756343001

jamisteven 8y ago

In other news, Amazons "Freudian Slip" Machine is 100% accurate.

deleted 8y ago

[deleted]

aurizon 8y ago

Only 28? Should be a complete 1:1, meaning they all seem crooked in one way or another...

alphabettsy 8y ago

Some commenters here mentioned they may have setup the test incorrectly and that may be, but I think the problem this highlights most is that technology used improperly, especially by law enforcement can have major ramifications and consequences. Is the contractor that makes software for your local department going to follow best practices and have the algorithm audited by experts? Will they release the code? Those are real considerations for any program that has the potential to help ruin someone’s life. It also possibly highlights the fact that algorithms are not immune to bias when they are designed by humans. Obviously I don’t think this bias is intentional, but so much of it isn’t and happens anyway.

dfundako 8y ago
I think a fun thought exercise is finding the fine line between tech and guns/alcohol/cars. You cannot sue a gun/car/alcohol manufacturer if their product is used to injure someone because it functioned as designed but was used maliciously. How does that legal precedent work when extrapolated to tech and something like facial recognition? If it worked exactly as designed and we know it has a margin of error (or can be used improperly and have disastrous results, like a car or gun), could Amazon or a tech administering it be liable for someone falsely imprisoned?
- vorpalhex 8y ago
  > could Amazon or a tech administering it be liable for someone falsely imprisoned? I would wager "absolutely not" for the exact same issue as firearms or cars. The person who mis-applied the technology or a middle man vendor though? Unless they fall under qualified immunity, there's your fall guy.
- guitarbill 8y ago
  It's still ethically questionable. In fact I'm struggling to come up with a better example than facial recognition tech (except other mass surveillance). Maybe cutting corners while developing driverless cars that results in the death of a pedestrian. Almost every engineering discipline has a code of ethics [0][1][2][3]. It's time software "engineering" grew up and did the same. I rarely see ethics mentioned on HN, and granted, people's view differ. But it's weird we're not having that conversation at all. [0] https://www.raeng.org.uk/policy/engineering-ethics/ethics https://www.raeng.org.uk/policy/engineering-ethics/ethics [1] https://www.ieee.org/about/corporate/governance/p7-8.html https://www.ieee.org/about/corporate/governance/p7-8.html [2] https://www.nspe.org/resources/ethics/code-ethics https://www.nspe.org/resources/ethics/code-ethics [3] http://www.asce.org/code-of-ethics/ http://www.asce.org/code-of-ethics/

ejlangev 8y ago

Amazon really has to take this back to the drawing board. ~5% accuracy is pretty terrible...

mc32 8y ago

There are definitely concerns about false positives, but it has to be compared to the current system. Are the results, effectiveness, better? When the gov steers opinion, we call it manufactured consent, when public advocacy organizations engage in sloppy methodology to further a cause, I propose calling it manufactured outrage.

drpgq 8y ago

I work in the industry and obviously I can't say for certain, but I don't think Amazon has anywhere near the most accurate face recognition software available. If you want to see where the tech is in terms of performance, the various NIST tests are highly informative.

exabrial 8y ago

Are they sure this was a mistake? I wouldn't be terribly surprised to find out if that many had mugshots.

reaperducer 8y ago

Well, it's not like police are using Amazon's face recognition technology in dragnets. Oh, wait: https://www.usatoday.com/story/tech/talkingtech/2018/06/29/capital-gazette-gunman-identified-using-facial-recognition-technology/744344002/ https://www.usatoday.com/story/tech/talkingtech/2018/06/29/c...

ericcumbee 8y ago

At the risk of being down voted....it recognizes 28 members of congress as criminals....i’d Call that a good start.

squozzer 8y ago

America is in too much danger to take any chances. The 28 positives should be held indefinitely until they can prove they're innocent. Presumption of Guilt is the new normal, isn't it?

andrewguenther 8y ago

> People of color were disproportionately falsely matched in our test. They are trying to make this racially charged without giving enough information to verify their claims. If you use a dataset of mugshots, that's statistically going to have more data on people of color. If you have more data on people of color, it is more likely to match people of color. Claiming the algorithm is racist because your data is racist is inflammatory bullshit.

spamizbad 8y ago
How is stating a fact- it disproportionately matched people of color- somehow the same as claiming racism?
- hoschicz 8y ago
  The fact is irrelevant to the problem and there is no reason to mention it, if they want the debate to stay professional and not emotional. The whole racism angle of the article seems way too loaded for me.
rrego 8y ago
Pretty lousy argument. Firstly, the majority of inmates in the US are white (58%). I'd assume mug shot stats are similar, or at the very least not mostly black people. Moreover, if a algorithm enforces bias to the detriment of inoccents, it's a bad algorithm
- andrewguenther 8y ago
  > Firstly, the majority of inmates in the US are white (58%). I'd assume mug shot stats are similar, or at the very least not mostly black people. I actually did not know this. Thank you. I had conflated incarceration rate with inmate population. If anyone is curious for a source, see here: https://www.bop.gov/about/statistics/statistics_inmate_race.jsp https://www.bop.gov/about/statistics/statistics_inmate_race.... > Moreover, if a algorithm enforces bias to the detriment of inoccents, it's a bad algorithm I agree with this 100%. But they haven't provided enough information about their dataset to make this conclusion. If they provided enough information for someone to independently analyze the data and reproduce the experiment, I would flip immediately.

j_lane 8y ago

maybe we should let the cops have this...

stretchwithme 8y ago

People have similar faces. Who knew? The technology is not perfect. And should never be used as evidence of a crime, just as an indication that two images might be of the same person and to have humans look at them. And then those humans will also make mistakes. People do look alike. And there are twins. I think a jury should require more evidence than just similar appearance. But such a match is a strong indication of where to look.

see-dante 8y ago
I don't think that's the point -- the problem isn't that "oh there are other processes that [...]," it's that this wave of technology is misused, is bias, and care must be taken when any of the signals we create. This is especially important as automation enters the equation.

knorker 8y ago

Sure. "Falsely" :-)

imnotlost 8y ago

The algo missed 507 of those crooks!