3 ms·

I can understand wanting to put heavy restrictions on the use of this technology, for example a rule that someone cannot be arrested based on software alone, an

I can understand wanting to put heavy restrictions on the use of this technology, for example a rule that someone cannot be arrested based on software alone, and a rule that it can only be used to help identify individuals that are clearly committing a crime. But it seems to me to be batshit insane to ban it completely. If we have video of, say, an armed robbery, and we (only) use software to get a suspect list before we have human beings looking at mug shots, what is the alleged downside?

cryptoz 5y ago
> If we have video of, say, an armed robbery, and we (only) use software to get a suspect list before we have human beings looking at mug shots, what is the alleged downside? Based on my reading of the article, the 'alleged' downside (this is very clearly real, there is no need to say alleged here!) is that minorities are being swept up in the system and wrongfully accused of crimes and then jailed. Your suggestion is, what if we didn't jail them? Just round up the innocently identified minorities and start investigating them? That is....hardly an improvement. In fact it would seem that while it might reduce falsely accused innocent people from going to jail, it could increase the number of racist incidents where police are harassing innocent minorities. And worst of all, it could further entrench racist decision making inside police departments. A much better suggestion in my opinion would be, test the software before deploying it so that it doesn't make racist decisions. Set the bar VERY high for this, so the software has to be really, really, really good. Otherwise don't use it.
- seanmcdirmid 5y ago
  > Based on my reading of the article, the 'alleged' downside (this is very clearly real, there is no need to say alleged here!) is that minorities are being swept up in the system and wrongfully accused of crimes and then jailed. Then is it really fair to call it facial recognition if it doesn't work? Or to say "we are banning this until it is more reliable" or something like that?
- godelski 5y ago
  To be clear, there are a lot more downsides too. Sci-Fi dystopians bring up facial recognition technology a lot and about how the technology can be abused and help a government rule with authoritarian power. I do think it is important to think about these issues since democracy isn't stable (Before we start a flame war: I'm not convinced any system is and I think you'd be hard pressed to argue one is). We should consider the vulnerabilities to our system of governance and this seems to be a common one: tracking citizens. But I think in our society "potential of authoritarian takeover" (sometimes called "turnkey tyranny") is less of a concern for people than racial bias (which is still an important topic, don't get me wrong). I think the difference is that one requires foresight and the other is an effect we can see right now. And of course, historically we've seen minorities become the early victims of authoritarians. We should think long and hard about these technologies and how to use them properly. Historically we haven't thought about ethics much in computer science, but the time isn't post hoc. We (Hacker News readers) are the experts, and we need to be talking about this decades ago, not decades from now. So let's not ignore the conversation.
rantwasp 5y ago
now assume you’re a racist pos that is also by chance enforcing the law. also assume that you’re allegedly going to use this as a pretext to investigate people that even though don’t appear on the footage sort of look like the persons in the footage (the software told you so!). now assume that based on your harassment you find something and you arrest those poor souls. does this seem far fetched? if you haven’t experienced this you may allegedly be privileged.
akomtu 5y ago
A small chance of being robbed is way better than a global survellience system that's tracking my face in all public places. Besides, criminals will simply learn to wear masks.
UnpossibleJim 5y ago
I won't lie, I truly understand the knee-jerk reaction to Orwellian surveillance by a police state. It's creepy, but is somewhat in place via semi constant NSA tracking that gets re-upped by private Senate hearings, so that's neither here nor there.... However, as far as facial recognition tracking goes, this doesn't seem like it needs to be an "All or Nothing" agreement, as you say Why can't we use facial recognition software with mugshots to narrow a list of suspects. Or use facial recognition software to enhance police work instead of doing a prosecutor's job. It's possible to ban facial recognition in certain arenas instead of alltogether.
cwkoss 5y ago
You're assuming that the technology works well and that people will understand its limitations and not gradually expand its application. The downside is that that assumption is an unrealistic fantasy. Innocent people will be charged with crimes by lazy cops who don't want to do the actual police work of thoroughly investigating suspect lists. There will be juries who don't understand prediction error and find innocent people guilty because they trust the "expert" who says the facial recognition system reported "99.99% accuracy". Some of the 40% of cops who have committed domestic violence will make unauthorized use of this system to hunt down fleeing battered spouses - and this unauthorized use will go unpunished. Innocent black people will be identified as subjects and arrested because of racial disparities in accuracy of these systems. Innocent political activists will be charged with crimes they didn't commit because a cop found video of a crime that the software says plausibly looks like the activist. Every time police get a new tool for establishing guilt, it is abused. The only way to make a facial recognition system that is just is if it is out of control of the police and it can only be used with a valid court warrant - and even then, I'm not sure that I trust the court's ability to control such a system securely.

cryptoz 5y ago

> If we have video of, say, an armed robbery, and we (only) use software to get a suspect list before we have human beings looking at mug shots, what is the alleged downside? Based on my reading of the article, the 'alleged' downside (this is very clearly real, there is no need to say alleged here!) is that minorities are being swept up in the system and wrongfully accused of crimes and then jailed. Your suggestion is, what if we didn't jail them? Just round up the innocently identified minorities and start investigating them? That is....hardly an improvement. In fact it would seem that while it might reduce falsely accused innocent people from going to jail, it could increase the number of racist incidents where police are harassing innocent minorities. And worst of all, it could further entrench racist decision making inside police departments. A much better suggestion in my opinion would be, test the software before deploying it so that it doesn't make racist decisions. Set the bar VERY high for this, so the software has to be really, really, really good. Otherwise don't use it.

seanmcdirmid 5y ago
> Based on my reading of the article, the 'alleged' downside (this is very clearly real, there is no need to say alleged here!) is that minorities are being swept up in the system and wrongfully accused of crimes and then jailed. Then is it really fair to call it facial recognition if it doesn't work? Or to say "we are banning this until it is more reliable" or something like that?
godelski 5y ago
To be clear, there are a lot more downsides too. Sci-Fi dystopians bring up facial recognition technology a lot and about how the technology can be abused and help a government rule with authoritarian power. I do think it is important to think about these issues since democracy isn't stable (Before we start a flame war: I'm not convinced any system is and I think you'd be hard pressed to argue one is). We should consider the vulnerabilities to our system of governance and this seems to be a common one: tracking citizens. But I think in our society "potential of authoritarian takeover" (sometimes called "turnkey tyranny") is less of a concern for people than racial bias (which is still an important topic, don't get me wrong). I think the difference is that one requires foresight and the other is an effect we can see right now. And of course, historically we've seen minorities become the early victims of authoritarians. We should think long and hard about these technologies and how to use them properly. Historically we haven't thought about ethics much in computer science, but the time isn't post hoc. We (Hacker News readers) are the experts, and we need to be talking about this decades ago, not decades from now. So let's not ignore the conversation.

rantwasp 5y ago

now assume you’re a racist pos that is also by chance enforcing the law. also assume that you’re allegedly going to use this as a pretext to investigate people that even though don’t appear on the footage sort of look like the persons in the footage (the software told you so!). now assume that based on your harassment you find something and you arrest those poor souls. does this seem far fetched? if you haven’t experienced this you may allegedly be privileged.

akomtu 5y ago

A small chance of being robbed is way better than a global survellience system that's tracking my face in all public places. Besides, criminals will simply learn to wear masks.

UnpossibleJim 5y ago

I won't lie, I truly understand the knee-jerk reaction to Orwellian surveillance by a police state. It's creepy, but is somewhat in place via semi constant NSA tracking that gets re-upped by private Senate hearings, so that's neither here nor there.... However, as far as facial recognition tracking goes, this doesn't seem like it needs to be an "All or Nothing" agreement, as you say Why can't we use facial recognition software with mugshots to narrow a list of suspects. Or use facial recognition software to enhance police work instead of doing a prosecutor's job. It's possible to ban facial recognition in certain arenas instead of alltogether.

cwkoss 5y ago

You're assuming that the technology works well and that people will understand its limitations and not gradually expand its application. The downside is that that assumption is an unrealistic fantasy. Innocent people will be charged with crimes by lazy cops who don't want to do the actual police work of thoroughly investigating suspect lists. There will be juries who don't understand prediction error and find innocent people guilty because they trust the "expert" who says the facial recognition system reported "99.99% accuracy". Some of the 40% of cops who have committed domestic violence will make unauthorized use of this system to hunt down fleeing battered spouses - and this unauthorized use will go unpunished. Innocent black people will be identified as subjects and arrested because of racial disparities in accuracy of these systems. Innocent political activists will be charged with crimes they didn't commit because a cop found video of a crime that the software says plausibly looks like the activist. Every time police get a new tool for establishing guilt, it is abused. The only way to make a facial recognition system that is just is if it is out of control of the police and it can only be used with a valid court warrant - and even then, I'm not sure that I trust the court's ability to control such a system securely.