5 ms·

Just so I have good arguments when talking to someone about this specific point, can you or someone else please go into more detail on why this will happen?

donohoe 7y ago
Not specific to facial recognition but covers it and the deeper issues. It is a thorough yet accessible primer on the many ways bias can creep into these systems: https://parametric.press/issue-01/the-myth-of-the-impartial-machine/ https://parametric.press/issue-01/the-myth-of-the-impartial-...
- masklinn 7y ago
  As @pinboard noted back in the dark ages of 2016, "machine learning is money laundering for bias".
paganel 7y ago
Not the OP by technology like this will explicitly make racial profiling legal again, because the power that be will hide behind the decisions taken by algorithms: “The computer told us to stop and search this black person’s car, it wasn’t us, the computer is smart, we were just following the computer’s orders”.
- bumby 7y ago
  I heard this referred to recently as the "automation fallacy"
- xena 7y ago
  As a transgender woman, this is my fear precisely.
- chr1 7y ago
  That's why there is https://en.wikipedia.org/wiki/Right_to_explanation https://en.wikipedia.org/wiki/Right_to_explanation
trevyn 7y ago
Along the lines of “everyone is breaking some law”, there could be a shift from “technically illegal but generally harmless and tolerated” to “illegal and mindlessly enforced by automated technology”. An example being China’s automated jaywalking-shamer.
- mc32 7y ago
  The only “good” outcome of this is we get to know how pervasive surveillance affects behavior and whether the benefits outweigh the detriments. In other words does it result in a better or worse society overall. Even if it’s better overall I’d it worth the negative effects on affected populations? It’s a living laboratory.
  - homonculus1 7y ago
    The benefits would have to be impossibly valuable to outweigh the constant burden and psychological damage of living in a totalitarian panopticon. Given that society functions without it and is constantly improving anyway--crime rates, poverty, mortality continue to drop from socioeconomic and technological development without the need for the intervention of pervasive surveillance--I personally literally cannot conceive of a benefit that would actually be worth it.
    - mc32 7y ago
      True, I do not see it turning out well. But it would be a laboratory into behavioral control. Something like this would eliminate needs for safe spaces, anti/hate speech, etc. because this would regulate all this anti social behavior in the first place.
bryanrasmussen 7y ago
There have been numerous issues where corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces and therefore be shut out of whatever the facial recognition is supposed to implement. For example facial recognition at a high priced building, suddenly non-white people can't get in because you did not show a real face at the scanner. Sorry, it's a bug not a feature. The same problem would of course apply to anyone with facial deformities etc. The problem would be essentially of two forms - Automatic exclusion from a group by facial feature, or automatic inclusion in a group by facial feature. Think of the automatic inclusion as advanced phrenology or other sort of woo, in this scenario company X sells the idea that they will have facial recognition of 'criminal types' based on whatever research they can pull out that criminals often correspond in appearance. Then they fill that corpus up with criminals which then is a nice self-selecting system of find the black/hispanic person and deny them the job, loan etc. because of likelihood for crime past the threshold set. the reason why this would happen is the same reason why banks used to deny loans to people in certain zip codes, because by doing so they could be racially profiling and excluding but pretend not to do so.
- mc32 7y ago
  >”corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces..” I’m not sure this is s good argument against the technology because there is a solution to this objection: they’ll improve it and ensure every face type is comparably recognized. It should be rejected on more basic principles.
  - rmrfrmrf 7y ago
    It's not as simple as "just make sure it works on everyone". Even if it were physically possible to capture the face of all 7 billion people on Earth, you'd start running into issues of false positives/negatives, overfitting, etc., and furthermore it would basically require every person to be periodically scanned from birth until death to ensure that the algorithm can recognize new faces.
lazyvoice 7y ago
For example, let's assume you live in Brunei where homosexuality is a capital crime. Facial-recognition (especially in concert with other pattern-recognition software) drastically reduces the man-power necessary to identify individuals for enforcement. If you're talking to someone who's more conservatively inclined I'd talk about how Iceland has managed to basically eliminate Down Syndrome. Prenatal pattern recognition has successfully convinced nearly all Icelandic parents who have a pregnancy where the child may have Down Syndrome to abort. If your listener doesn't care about gay rights AND thinks any fetus with a higher likelihood of having Down Syndrome should be aborted, then they're probably exactly the sort of people working on or hoping to use these tools.
- mnw21cam 7y ago
  Okay, I see your Down Syndrome point, and raise you Autism/Asperger Syndrome. Alright, there isn't currently a test that would enable people with Autism/Asperger Syndrome to be systematically aborted, but it isn't so hard to imagine a future where it would be possible. Given the high proportion of people in this forum with these conditions, how do we feel about this?
  - phissk 7y ago
    Please note that you're not aborting 'people' ... just a thing that might become a person. You're giving (actual) people the opportunity to choose to have healthier families. - Note that I say 'healthier families' specifically because willfully passing on genetic features that disadvantage your children burdens not just them but your entire family unit.
    - mistersquid 7y ago
      The problem lies within the generalizations presented by the words “healthier” and “burdens”. Many families with Downs children describe how much joy and love their Downs children bring to the family, how the family members learned to cherish what’s important in life, etc. These families compared to many families with non-Downs children are quite healthy and arguably not burdened though they experience different challenges than other families. EDIT: fix quote; change last preposition in second sentence.

donohoe 7y ago

Not specific to facial recognition but covers it and the deeper issues. It is a thorough yet accessible primer on the many ways bias can creep into these systems: https://parametric.press/issue-01/the-myth-of-the-impartial-machine/ https://parametric.press/issue-01/the-myth-of-the-impartial-...

masklinn 7y ago
As @pinboard noted back in the dark ages of 2016, "machine learning is money laundering for bias".

paganel 7y ago

Not the OP by technology like this will explicitly make racial profiling legal again, because the power that be will hide behind the decisions taken by algorithms: “The computer told us to stop and search this black person’s car, it wasn’t us, the computer is smart, we were just following the computer’s orders”.

bumby 7y ago
I heard this referred to recently as the "automation fallacy"
xena 7y ago
As a transgender woman, this is my fear precisely.
chr1 7y ago
That's why there is https://en.wikipedia.org/wiki/Right_to_explanation https://en.wikipedia.org/wiki/Right_to_explanation

trevyn 7y ago

Along the lines of “everyone is breaking some law”, there could be a shift from “technically illegal but generally harmless and tolerated” to “illegal and mindlessly enforced by automated technology”. An example being China’s automated jaywalking-shamer.

mc32 7y ago
The only “good” outcome of this is we get to know how pervasive surveillance affects behavior and whether the benefits outweigh the detriments. In other words does it result in a better or worse society overall. Even if it’s better overall I’d it worth the negative effects on affected populations? It’s a living laboratory.
- homonculus1 7y ago
  The benefits would have to be impossibly valuable to outweigh the constant burden and psychological damage of living in a totalitarian panopticon. Given that society functions without it and is constantly improving anyway--crime rates, poverty, mortality continue to drop from socioeconomic and technological development without the need for the intervention of pervasive surveillance--I personally literally cannot conceive of a benefit that would actually be worth it.
  - mc32 7y ago
    True, I do not see it turning out well. But it would be a laboratory into behavioral control. Something like this would eliminate needs for safe spaces, anti/hate speech, etc. because this would regulate all this anti social behavior in the first place.

bryanrasmussen 7y ago

There have been numerous issues where corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces and therefore be shut out of whatever the facial recognition is supposed to implement. For example facial recognition at a high priced building, suddenly non-white people can't get in because you did not show a real face at the scanner. Sorry, it's a bug not a feature. The same problem would of course apply to anyone with facial deformities etc. The problem would be essentially of two forms - Automatic exclusion from a group by facial feature, or automatic inclusion in a group by facial feature. Think of the automatic inclusion as advanced phrenology or other sort of woo, in this scenario company X sells the idea that they will have facial recognition of 'criminal types' based on whatever research they can pull out that criminals often correspond in appearance. Then they fill that corpus up with criminals which then is a nice self-selecting system of find the black/hispanic person and deny them the job, loan etc. because of likelihood for crime past the threshold set. the reason why this would happen is the same reason why banks used to deny loans to people in certain zip codes, because by doing so they could be racially profiling and excluding but pretend not to do so.

mc32 7y ago
>”corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces..” I’m not sure this is s good argument against the technology because there is a solution to this objection: they’ll improve it and ensure every face type is comparably recognized. It should be rejected on more basic principles.
- rmrfrmrf 7y ago
  It's not as simple as "just make sure it works on everyone". Even if it were physically possible to capture the face of all 7 billion people on Earth, you'd start running into issues of false positives/negatives, overfitting, etc., and furthermore it would basically require every person to be periodically scanned from birth until death to ensure that the algorithm can recognize new faces.

lazyvoice 7y ago

For example, let's assume you live in Brunei where homosexuality is a capital crime. Facial-recognition (especially in concert with other pattern-recognition software) drastically reduces the man-power necessary to identify individuals for enforcement. If you're talking to someone who's more conservatively inclined I'd talk about how Iceland has managed to basically eliminate Down Syndrome. Prenatal pattern recognition has successfully convinced nearly all Icelandic parents who have a pregnancy where the child may have Down Syndrome to abort. If your listener doesn't care about gay rights AND thinks any fetus with a higher likelihood of having Down Syndrome should be aborted, then they're probably exactly the sort of people working on or hoping to use these tools.

mnw21cam 7y ago
Okay, I see your Down Syndrome point, and raise you Autism/Asperger Syndrome. Alright, there isn't currently a test that would enable people with Autism/Asperger Syndrome to be systematically aborted, but it isn't so hard to imagine a future where it would be possible. Given the high proportion of people in this forum with these conditions, how do we feel about this?
- phissk 7y ago
  Please note that you're not aborting 'people' ... just a thing that might become a person. You're giving (actual) people the opportunity to choose to have healthier families. - Note that I say 'healthier families' specifically because willfully passing on genetic features that disadvantage your children burdens not just them but your entire family unit.
  - mistersquid 7y ago
    The problem lies within the generalizations presented by the words “healthier” and “burdens”. Many families with Downs children describe how much joy and love their Downs children bring to the family, how the family members learned to cherish what’s important in life, etc. These families compared to many families with non-Downs children are quite healthy and arguably not burdened though they experience different challenges than other families. EDIT: fix quote; change last preposition in second sentence.