4 ms·

Neither is acceptable in my eyes. Implicit in your post is the idea that the system functions and functions well. If a single person is incorrectly identified

Neither is acceptable in my eyes. Implicit in your post is the idea that the system functions and functions well. If a single person is incorrectly identified by a system design to catch warrants, the system has failed beyond redemption. No system can guarantee such perfect recognition, so the only reasonable thing to do is never deploy them.

zo1 6y ago
And implicit in your post is that this system can't be designed to function well at all. A logical view would be that if something isn't "perfect", you build around that imperfection and use it for its good parts whilst mitigating and eliminating the bad parts.
- Pfhreak 6y ago
  No, it's explicit in my post. A facial recognition system cannot be designed to function well. A logical view says that anything short of perfect will absolutely ruin people's lives.
  - throwawaysea 6y ago
    No, it will improve society overall because it will lead to significantly higher accountability and consequences for criminals who are caught. There is no logic or evidence to your claim that “anything short of perfect will absolutely ruin people's lives”. We already live in an imperfect world. It will never be perfect. Facial recognition with humans in the loop will lead to a far higher ratio of bad people facing consequences to good people incorrectly identified. It is a net improvement for society and it is much more reasonable to claim it will “absolutely improve people’s lives”.
    - Pfhreak 6y ago
      There is absolutely logic in it. If you get falsely arrested (not even convicted), you can lose your job, get deported, spend thousands trying to clear your name, or even just get shot by the police. AI tech will have false positives. Technology that allows officers to screen many many people at scale will result in false positive arrests (or police violence) that would not have happened otherwise. Therefore, this technology will ruin lives. This is clearly logical. You might be ok with that cost, I'm not. But that's a different line of debate. I believe the idea that it is a net improvement is the idea that is unbelievable and unfounded given the current trends in AI. What gives you confidence?
      - throwawaysea 6y ago
        Again there is a human in the loop to validate a potential match, before officers get dispatched. And if the dispatcher doesn't do it, we can easily just have the police officer responding to validate the match between the surveillance photo and a known photo of the suspect. Humans already have a false positive rate as part of existing policing procedures, and facial recognition doesn’t make the rate worse as long as it retains a human validating the AI’s match. Also, almost every state has a process to expunge arrests even IF you were convicted (of something minor). As for why I’m confident about net improvement - it’s because the amount of true positive arrests (and subsequent convictions) will be far greater than false positive arrests. Therefore relative to the status quo it’ll be a net positive of criminals facing consequences, victims being compensated or getting closure, subsequent reduction in crime through deterrence, etc.
    - odessacubbage 6y ago
      what about an incorrect flagging being used as evidence in a case in order to prove that a suspect was in a given location around the time that an offense occurred? do we render the entire data set as inadmissible in court for the risk of such misclassifications? >anything short of perfect will absolutely ruin people's lives this is more or less the basis for all thought within our system of justice.
lr4444lr 6y ago
If a person is incorrectly identified as the target of an open warrant, then as soon as he shows proof of ID, he's free to go and the system is corrected to flag him as a false positive, not only never bothering him again for this search, but probably making him less likely to ever be falsely picked up again. If we can't assume that any public system or institution is capable of functioning well or being corrected, we might as well have no law enforcement whatsoever.
- Pfhreak 6y ago
  You have a very rose colored view of the world. What will happen is the police will consider the person dangerous, approach cautiously or aggressively, cuff them, throw them into the back of a cruiser and then maybe check their id. And you are assuming that all people have ID on them. Or are even here legally. "Show me your papers" isn't better because a computer told the cops who to target.
okthenletssee 6y ago
Computers are only a tool. Humans have to make the final decisions. Under those circumstances, what exactly is the problem? I notice that all the articles warning about AI supervision never mention the error rate of humans. Humans misidentify people, too.
- Pfhreak 6y ago
  Humans cannot scale the same way AI can. AI can easily scan every face at a football game. Humans can't. AI could easily scale to watch everyone who walks through every intersection downtown. Humans can't. If AI makes a much smaller percentage of mistakes (which is itself a questionable thesis), it will still make orders of magnitude more mistakes because it will be making orders of magnitude more evaluations. Also, for better or for worse, people tend to trust that computers are 'logical' and correct. This is especially troubling because a human can explain their mistakes ("the person was wearing similar clothing, had a limp, etc."), while AI generally can't give you a reason that you can easily understand. > Computers are only a tool. Humans have to make the final decisions. Under those circumstances, what exactly is the problem? Humans are bad at evaluating computing systems. Have you heard the stories of people driving their cars into lakes because a GPS told them to? Now imagine the line was even fuzzier ("We think this could be our suspect, but we're not sure. The computer positively ID'd them though...")
  - trhway 6y ago
    well, sci-fi and everybody else was warning humans about that since like mid-20th century, and yet the human race is as unprepared as one can be (similarly to the climate change situation), and still puts the major hope into the "putting the genie back into the bottle" strategy which is obviously not going to work... Kind or reminds of the lockdowns as a strategy to fight the covid :)
  - okthenletssee 6y ago
    Even if the AI identifies many people, there still have to be humans checking them out. The limited number of humans may have more time to check each suspect out than they would have without the AI, because the AI takes over some of the work. Humans can be aware of the limitations of "AI". It's very rare that somebody drives into a lake because of GPS failures.
throwawaysea 6y ago
All facial recognition implementations in US policing have a human in the loop to verify the match. It introduces no more error than already exists when we use human judgment to spot and catch suspects by random luck using beat cops. This is better because it narrows down the search for suspects and uses limited staff efficiently, leading to an overall safer society. All this fear mongering about false positives is irrelevant because there is a human in the loop. > And you are assuming that all people have ID on them. Or are even here legally. Sorry but almost everyone does have ID on them at all times. It is unreasonable to design our society for those who don’t. As for those here illegally - well perhaps they shouldn’t be here illegally. Again, we shouldn’t design our society that is based on laws around them. > If a single person is incorrectly identified by a system design to catch warrants, the system has failed beyond redemption. I disagree. Any large scale and complex system will be imperfect. Your argument is basically “letting perfect be the enemy of good”.
- Pfhreak 6y ago
  > Sorry but almost everyone does have ID on them at all times. This is objectively false. 11+% of Americans don't have a government issued photo id. Presumably many more forget their id on any given day. The folks who don't have IDs are disproportionately minorities and the elderly. In some places, as few as 55% off eligible Black people had IDs. Millions more are here illegally, and yes, we should design our society to accommodate them. "Maybe they shouldn't be here" is both callous and unrealistic. No one should have to live in fear that they'll be spotted on some camera and have their life ruined. > All this fear mongering about false positives is irrelevant because there is a human in the loop. Humans in the loop won't fix failures. Last week we were all discussing how hospital administrators blamed "the algorithm" when vaccines were issued to admins rather than front line workers. People constantly say, "well, the computer says..." when the computer is obviously making a mistake. The TSA regularly screws up people's travel because they make mistakes about the no fly list. A human can be in the loop, but in practice the human isn't going to meaningfully question the outcome.
  - throwawaysea 6y ago
    > Millions more are here illegally, and yes, we should design our society to accommodate them. "Maybe they shouldn't be here" is both callous and unrealistic. No one should have to live in fear that they'll be spotted on some camera and have their life ruined. Those that break the law SHOULD have to live in fear that they'll face consequences. In some cases that may amount to having "their life ruined" and in other cases the consequences may not be that dramatic. But we have a society built on laws, and enforcing the law is required in a fair and just society. > Last week we were all discussing how hospital administrators blamed "the algorithm" when vaccines were issued to admins rather than front line workers. People constantly say, "well, the computer says..." when the computer is obviously making a mistake. In my opinion, you are comparing very different situations. The hospital admins have an incentive to get admins vaccinated first, and are not in a position where they're tasked with validating an algorithm's match. Additionally, the algorithm in question did not involve machine learning as a facial recognition algorithm would. It was simply software implementing a very straightforward formula (https://www.technologyreview.com/2020/12/21/1015303/stanford-vaccine-algorithm/ https://www.technologyreview.com/2020/12/21/1015303/stanford...). > The TSA regularly screws up people's travel because they make mistakes about the no fly list. This isn't because of facial recognition or black box machine learning algorithms. It is because the TSA uses very rudimentary heuristics like name matching, to err on the side of safety. I disagree with the unsophisticated approach they use, for what its worth. A complex AI based matching algorithm would do a better job of balancing safety against such mistakes.

zo1 6y ago

And implicit in your post is that this system can't be designed to function well at all. A logical view would be that if something isn't "perfect", you build around that imperfection and use it for its good parts whilst mitigating and eliminating the bad parts.

Pfhreak 6y ago
No, it's explicit in my post. A facial recognition system cannot be designed to function well. A logical view says that anything short of perfect will absolutely ruin people's lives.
- throwawaysea 6y ago
  No, it will improve society overall because it will lead to significantly higher accountability and consequences for criminals who are caught. There is no logic or evidence to your claim that “anything short of perfect will absolutely ruin people's lives”. We already live in an imperfect world. It will never be perfect. Facial recognition with humans in the loop will lead to a far higher ratio of bad people facing consequences to good people incorrectly identified. It is a net improvement for society and it is much more reasonable to claim it will “absolutely improve people’s lives”.
  - Pfhreak 6y ago
    There is absolutely logic in it. If you get falsely arrested (not even convicted), you can lose your job, get deported, spend thousands trying to clear your name, or even just get shot by the police. AI tech will have false positives. Technology that allows officers to screen many many people at scale will result in false positive arrests (or police violence) that would not have happened otherwise. Therefore, this technology will ruin lives. This is clearly logical. You might be ok with that cost, I'm not. But that's a different line of debate. I believe the idea that it is a net improvement is the idea that is unbelievable and unfounded given the current trends in AI. What gives you confidence?
    - throwawaysea 6y ago
      Again there is a human in the loop to validate a potential match, before officers get dispatched. And if the dispatcher doesn't do it, we can easily just have the police officer responding to validate the match between the surveillance photo and a known photo of the suspect. Humans already have a false positive rate as part of existing policing procedures, and facial recognition doesn’t make the rate worse as long as it retains a human validating the AI’s match. Also, almost every state has a process to expunge arrests even IF you were convicted (of something minor). As for why I’m confident about net improvement - it’s because the amount of true positive arrests (and subsequent convictions) will be far greater than false positive arrests. Therefore relative to the status quo it’ll be a net positive of criminals facing consequences, victims being compensated or getting closure, subsequent reduction in crime through deterrence, etc.
  - odessacubbage 6y ago
    what about an incorrect flagging being used as evidence in a case in order to prove that a suspect was in a given location around the time that an offense occurred? do we render the entire data set as inadmissible in court for the risk of such misclassifications? >anything short of perfect will absolutely ruin people's lives this is more or less the basis for all thought within our system of justice.

lr4444lr 6y ago

If a person is incorrectly identified as the target of an open warrant, then as soon as he shows proof of ID, he's free to go and the system is corrected to flag him as a false positive, not only never bothering him again for this search, but probably making him less likely to ever be falsely picked up again. If we can't assume that any public system or institution is capable of functioning well or being corrected, we might as well have no law enforcement whatsoever.

Pfhreak 6y ago
You have a very rose colored view of the world. What will happen is the police will consider the person dangerous, approach cautiously or aggressively, cuff them, throw them into the back of a cruiser and then maybe check their id. And you are assuming that all people have ID on them. Or are even here legally. "Show me your papers" isn't better because a computer told the cops who to target.

okthenletssee 6y ago

Computers are only a tool. Humans have to make the final decisions. Under those circumstances, what exactly is the problem? I notice that all the articles warning about AI supervision never mention the error rate of humans. Humans misidentify people, too.

Pfhreak 6y ago
Humans cannot scale the same way AI can. AI can easily scan every face at a football game. Humans can't. AI could easily scale to watch everyone who walks through every intersection downtown. Humans can't. If AI makes a much smaller percentage of mistakes (which is itself a questionable thesis), it will still make orders of magnitude more mistakes because it will be making orders of magnitude more evaluations. Also, for better or for worse, people tend to trust that computers are 'logical' and correct. This is especially troubling because a human can explain their mistakes ("the person was wearing similar clothing, had a limp, etc."), while AI generally can't give you a reason that you can easily understand. > Computers are only a tool. Humans have to make the final decisions. Under those circumstances, what exactly is the problem? Humans are bad at evaluating computing systems. Have you heard the stories of people driving their cars into lakes because a GPS told them to? Now imagine the line was even fuzzier ("We think this could be our suspect, but we're not sure. The computer positively ID'd them though...")
- trhway 6y ago
  well, sci-fi and everybody else was warning humans about that since like mid-20th century, and yet the human race is as unprepared as one can be (similarly to the climate change situation), and still puts the major hope into the "putting the genie back into the bottle" strategy which is obviously not going to work... Kind or reminds of the lockdowns as a strategy to fight the covid :)
- okthenletssee 6y ago
  Even if the AI identifies many people, there still have to be humans checking them out. The limited number of humans may have more time to check each suspect out than they would have without the AI, because the AI takes over some of the work. Humans can be aware of the limitations of "AI". It's very rare that somebody drives into a lake because of GPS failures.

throwawaysea 6y ago

All facial recognition implementations in US policing have a human in the loop to verify the match. It introduces no more error than already exists when we use human judgment to spot and catch suspects by random luck using beat cops. This is better because it narrows down the search for suspects and uses limited staff efficiently, leading to an overall safer society. All this fear mongering about false positives is irrelevant because there is a human in the loop. > And you are assuming that all people have ID on them. Or are even here legally. Sorry but almost everyone does have ID on them at all times. It is unreasonable to design our society for those who don’t. As for those here illegally - well perhaps they shouldn’t be here illegally. Again, we shouldn’t design our society that is based on laws around them. > If a single person is incorrectly identified by a system design to catch warrants, the system has failed beyond redemption. I disagree. Any large scale and complex system will be imperfect. Your argument is basically “letting perfect be the enemy of good”.

Pfhreak 6y ago
> Sorry but almost everyone does have ID on them at all times. This is objectively false. 11+% of Americans don't have a government issued photo id. Presumably many more forget their id on any given day. The folks who don't have IDs are disproportionately minorities and the elderly. In some places, as few as 55% off eligible Black people had IDs. Millions more are here illegally, and yes, we should design our society to accommodate them. "Maybe they shouldn't be here" is both callous and unrealistic. No one should have to live in fear that they'll be spotted on some camera and have their life ruined. > All this fear mongering about false positives is irrelevant because there is a human in the loop. Humans in the loop won't fix failures. Last week we were all discussing how hospital administrators blamed "the algorithm" when vaccines were issued to admins rather than front line workers. People constantly say, "well, the computer says..." when the computer is obviously making a mistake. The TSA regularly screws up people's travel because they make mistakes about the no fly list. A human can be in the loop, but in practice the human isn't going to meaningfully question the outcome.
- throwawaysea 6y ago
  > Millions more are here illegally, and yes, we should design our society to accommodate them. "Maybe they shouldn't be here" is both callous and unrealistic. No one should have to live in fear that they'll be spotted on some camera and have their life ruined. Those that break the law SHOULD have to live in fear that they'll face consequences. In some cases that may amount to having "their life ruined" and in other cases the consequences may not be that dramatic. But we have a society built on laws, and enforcing the law is required in a fair and just society. > Last week we were all discussing how hospital administrators blamed "the algorithm" when vaccines were issued to admins rather than front line workers. People constantly say, "well, the computer says..." when the computer is obviously making a mistake. In my opinion, you are comparing very different situations. The hospital admins have an incentive to get admins vaccinated first, and are not in a position where they're tasked with validating an algorithm's match. Additionally, the algorithm in question did not involve machine learning as a facial recognition algorithm would. It was simply software implementing a very straightforward formula (https://www.technologyreview.com/2020/12/21/1015303/stanford-vaccine-algorithm/ https://www.technologyreview.com/2020/12/21/1015303/stanford...). > The TSA regularly screws up people's travel because they make mistakes about the no fly list. This isn't because of facial recognition or black box machine learning algorithms. It is because the TSA uses very rudimentary heuristics like name matching, to err on the side of safety. I disagree with the unsophisticated approach they use, for what its worth. A complex AI based matching algorithm would do a better job of balancing safety against such mistakes.