5 ms·

I agree that automated systems applied to the same task will (eventually) beat human performance by a lot, but there are 2 issues that make this less relevant

by gota 7y ago

I agree that automated systems applied to the same task will (eventually) beat human performance by a lot, but there are 2 issues that make this less relevant in this case. The first is one relates to this application of facial recognition targeting poor people. This woman had no documentation on her. I'd wager this is really common in Brazil. How could she possibly prove she is _NOT_ the person the software with 99.999..% accuracy says she is? If the actual criminal wasn't already in jail, would she be set free in the same day she was arrested? The second issue is perhaps more universal to other automations, but certainly potentializes the first one, dealing with _scale_. Human cops can't check every person they see against the entire database of wanted people. 25% of false-positives from every cop is a hell of a lot less than 0.00001% of an automated system when applied to every crowd, everywhere, all the time.

gtirloni 7y ago
For the first issue, carry your documents. I've been taught that since I first got my ID at 12yo. My understanding is that the automation is to verify people who are needed by law, right? They don't voluntarily hear a beep and go into the jail. There are police officers that still have to arrest them and check their documents. The situation is so bad in Rio that society simply will have to deal with any deficiencies in the automated system. It will still be better than the current reality. The fact that police officers are probably not giving people of color enough flexibility in situations like these is beyond the scope of the automation system. EDIT: Carrying your documents is not a popular opinion among the US audience, I understand. Still, I find it very reasonable and I think the situation that happened to this woman is being blown out of proportion.
- droithomme 7y ago
  The US tries not to be a papers please police state where it's a crime not to carry papers with you and present them on demand. We consider systems like that to be totalitarian and despotic based on our past experiences observing and waging war against hostile oppressive nations with such systems.
  - patrick5415 7y ago
    He/she is not making this up. This is pretty much exactly how I think. When somebody says, “don’t want to be arrested by a bot? Just carry you papers!”, that sounds like absolutely evil tyranny to me.
  - foofoo55 7y ago
    > We consider systems like that to be totalitarian Who is "we"? Non-citizens of the USA are required to carry their immigration status documents with them, [1] and checkpoints within the 100-mile border zone (where 2/3 of the population lives [2]) are one place where they must be presented. If you are a US citizen without proof, and are stopped, and the official thinks you are lying, I believe you are going to have A Bad Day. [1] https://www.aclu.org/blog/immigrants-rights/immigrants-rights-and-detention/your-rights-border-zone https://www.aclu.org/blog/immigrants-rights/immigrants-right... [2] https://www.aclu.org/other/constitution-100-mile-border-zone https://www.aclu.org/other/constitution-100-mile-border-zone
    - AdrianB1 7y ago
      Well, in most cases "we" means the permanent legal inhabitants, not tourists. Legislation is not created by tourists.
      - facorreia 7y ago
        Permanent legal residents are required to carry their IDs.
    - wcoenen 7y ago
      > If you are a US citizen without proof, and are stopped, and the official thinks you are lying, I believe you are going to have A Bad Day. e.g. US citizen Valeria Alvarado was shot and killed when she panicked and tried to get away from plainclothes CBP agents.
  - deleted 7y ago
    [deleted]
  - 7y ago
- NikolaNovak 7y ago
  Indeed, notion of carrying your documents is very culturally dependent, from a rational decision to quasi-religious fervently-held belief. Coming from Eastern Europe originally, I carry my ID everywhere as a matter of habit in Canada; but in turn get very very very nervous and uncomfortable if it's taken from me (for example, at hotels in some countries, or at the border for a quick check).
frogpelt 7y ago
Scalability is definitely the problem here.
deleted 7y ago
[deleted]
dragonwriter 7y ago
> How could she possibly prove she is _NOT_ the person the software with 99.999..% accuracy says she is? She couldn't prove it any more of the source of the erroneous identification was a human looking at a wanted picture and matching her against it. Any reasonable criminal justice system incorporates the fact that innocent people will be arrested because it's impractical to impose high proof requirements for arrest, but at the same time will provide due process and have high proof requirements for conviction. If a justice system has a problem on either of those dimensions z they need urgently to be addressed independently of the use of facial recognition as an input to arrest decisions.
droithomme 7y ago
> 99.999% accuracy OK, so for a given facial profile, 0.001% of people passing in a crowd will be incorrectly identified as a match. That's 1 in 100,000 people who pass by will be incorrectly matched against a particular profile, correct? How many people walk past this camera? 1000 in a day? How many cameras are deployed in an area? How many facial profiles does it have in its database that it is checking against? It seems to me that if you have a lot of faces you are looking for and a big crowd, or even a modest crowd, then you are going to get a lot of false positives. Now if the wanted person walks past the camera then it has a chance to get its identification correct. If they don't walk past that camera there's no chance to get its identification correct. A given streetcorner with a camera where 1000 people walk past in a day may be in a city with 5 million people, only one of whom is the wanted person, and who might never pass that streetcorner. Given all this, if a match is triggered, what are the actual odds that the person is the right person? Is it more than 1 in 100? More than 1 in 1000? Does an officer have probable cause for an arrest if there is a 0.1% chance someone has done something? Do they have reasonable suspicion for a detention? Is the officer justified in shooting and killing the person who has had a facial match if they attempt to arrest the person and the person runs away?
- EForEndeavour 7y ago
  Maybe an unexpected benefit of AI-augmented law enforcement is that police, lawyers, and judges will be required to learn (more) applied statistics.

by gota 7y ago

gtirloni 7y ago

For the first issue, carry your documents. I've been taught that since I first got my ID at 12yo. My understanding is that the automation is to verify people who are needed by law, right? They don't voluntarily hear a beep and go into the jail. There are police officers that still have to arrest them and check their documents. The situation is so bad in Rio that society simply will have to deal with any deficiencies in the automated system. It will still be better than the current reality. The fact that police officers are probably not giving people of color enough flexibility in situations like these is beyond the scope of the automation system. EDIT: Carrying your documents is not a popular opinion among the US audience, I understand. Still, I find it very reasonable and I think the situation that happened to this woman is being blown out of proportion.

droithomme 7y ago
The US tries not to be a papers please police state where it's a crime not to carry papers with you and present them on demand. We consider systems like that to be totalitarian and despotic based on our past experiences observing and waging war against hostile oppressive nations with such systems.
- patrick5415 7y ago
  He/she is not making this up. This is pretty much exactly how I think. When somebody says, “don’t want to be arrested by a bot? Just carry you papers!”, that sounds like absolutely evil tyranny to me.
- foofoo55 7y ago
  > We consider systems like that to be totalitarian Who is "we"? Non-citizens of the USA are required to carry their immigration status documents with them, [1] and checkpoints within the 100-mile border zone (where 2/3 of the population lives [2]) are one place where they must be presented. If you are a US citizen without proof, and are stopped, and the official thinks you are lying, I believe you are going to have A Bad Day. [1] https://www.aclu.org/blog/immigrants-rights/immigrants-rights-and-detention/your-rights-border-zone https://www.aclu.org/blog/immigrants-rights/immigrants-right... [2] https://www.aclu.org/other/constitution-100-mile-border-zone https://www.aclu.org/other/constitution-100-mile-border-zone
  - AdrianB1 7y ago
    Well, in most cases "we" means the permanent legal inhabitants, not tourists. Legislation is not created by tourists.
    - facorreia 7y ago
      Permanent legal residents are required to carry their IDs.
  - wcoenen 7y ago
    > If you are a US citizen without proof, and are stopped, and the official thinks you are lying, I believe you are going to have A Bad Day. e.g. US citizen Valeria Alvarado was shot and killed when she panicked and tried to get away from plainclothes CBP agents.
- deleted 7y ago
  [deleted]
- 7y ago
NikolaNovak 7y ago
Indeed, notion of carrying your documents is very culturally dependent, from a rational decision to quasi-religious fervently-held belief. Coming from Eastern Europe originally, I carry my ID everywhere as a matter of habit in Canada; but in turn get very very very nervous and uncomfortable if it's taken from me (for example, at hotels in some countries, or at the border for a quick check).

frogpelt 7y ago

Scalability is definitely the problem here.

deleted 7y ago

[deleted]

dragonwriter 7y ago

> How could she possibly prove she is _NOT_ the person the software with 99.999..% accuracy says she is? She couldn't prove it any more of the source of the erroneous identification was a human looking at a wanted picture and matching her against it. Any reasonable criminal justice system incorporates the fact that innocent people will be arrested because it's impractical to impose high proof requirements for arrest, but at the same time will provide due process and have high proof requirements for conviction. If a justice system has a problem on either of those dimensions z they need urgently to be addressed independently of the use of facial recognition as an input to arrest decisions.

droithomme 7y ago

> 99.999% accuracy OK, so for a given facial profile, 0.001% of people passing in a crowd will be incorrectly identified as a match. That's 1 in 100,000 people who pass by will be incorrectly matched against a particular profile, correct? How many people walk past this camera? 1000 in a day? How many cameras are deployed in an area? How many facial profiles does it have in its database that it is checking against? It seems to me that if you have a lot of faces you are looking for and a big crowd, or even a modest crowd, then you are going to get a lot of false positives. Now if the wanted person walks past the camera then it has a chance to get its identification correct. If they don't walk past that camera there's no chance to get its identification correct. A given streetcorner with a camera where 1000 people walk past in a day may be in a city with 5 million people, only one of whom is the wanted person, and who might never pass that streetcorner. Given all this, if a match is triggered, what are the actual odds that the person is the right person? Is it more than 1 in 100? More than 1 in 1000? Does an officer have probable cause for an arrest if there is a 0.1% chance someone has done something? Do they have reasonable suspicion for a detention? Is the officer justified in shooting and killing the person who has had a facial match if they attempt to arrest the person and the person runs away?

EForEndeavour 7y ago
Maybe an unexpected benefit of AI-augmented law enforcement is that police, lawyers, and judges will be required to learn (more) applied statistics.