11 ms·

Defund Facial Recognition Before It's Too Late

23 points by sneeze-slayer 6y ago

core-questions 6y ago
> Rooted in discredited pseudoscience and racist eugenics theories that claim to use facial structure and head shape to assess mental capacity and character, automated facial-recognition software uses artificial intelligence, machine learning, and other forms of modern computing to capture the details of people’s faces and compare that information to existing photo databases with the goal of identifying, verifying, categorizing, and locating people. I'm sorry, what? Does this person think that phrenology / physiognomy, two old pseudosciences that have been discredited for a hundred years or more, are actually at play within ML systems? I'm totally willing to believe that ML facial recognition systems insufficiently trained on a wide enough set of faces will mistake one person for another. Sure. But to pretend that the system is based on eugenics belies a critical lack of understanding of what these things actually do, and ascribes agency and racial animus to a computer program. Pretty clear to me that this shows the author doesn't really know anything about how these things work. The reason to not want facial recognition in public spaces is the same as to not want mass surveillance: a reasonable expectation of privacy by citizens. Of course, if also want no police in these areas at the same time, one should not be surprised if eventually they go from peaceable public squares, to a haven for petty crime, to eventually a fearful place people avoid.
- joshuamorton 6y ago
  > I'm sorry, what? Does this person think that phrenology / physiognomy, two old pseudosciences that have been discredited for a hundred years or more, are actually at play within ML systems? https://callingbullshit.org/case_studies/case_study_criminal_machine_learning.html https://callingbullshit.org/case_studies/case_study_criminal... This shouldn't be a problem, because as you note phrenology is pseudoscience that's been discredited for over a century. And yet. To the broader point, I'd argue that in generally any attempt to predict criminality from facial structure/face picture is phrenological in nature. And people who do know what's at play with ML systems do agree with this take: https://twitter.com/ylecun/status/1276147230295166984 https://twitter.com/ylecun/status/1276147230295166984
  - core-questions 6y ago
    > And yet. Well, what's happening there is not a study of phrenology at all (which posited specific regions of facial/skull structure being indicators). It's actually a very interesting thing to look at. There was a previous one that was reporting some degree of success in determining whether someone was homosexual via ML. Here's the thing: if this turns out to have actual predictive power, then it's a subject worthy of scientific study, whether you like the outcomes and conclusions or not. Plenty of other worthy areas of endeavour (e.g. psychometric IQ research) have also revealed uncomfortable truths. If instead these things turn out to not have any legitimate research value (i.e. can't make predictions that can be experimentally verified), then we can stop looking at them, but as long as they continue to maintain a relatively consistent relationship to observable reality, they're as worthy a form of science as any other anthropological research is. We have a choice either face this head on, include it in policies, and build our sociology around the truth, or we can put our heads in the sand to make people feel better. I for one believe that truth is far more important than feelings, and that if we had continually given higher credence to feelings the Enlightenment and most scientific progress we've had would have been far slower if it had happened at all.
    - joshuamorton 6y ago
      > one that was reporting some degree of success in determining whether someone was homosexual via ML Yes, which was also phrenological in nature. > one that was reporting some degree of success in determining whether someone was homosexual via ML I've yet to see any "uncomfortable truth" from psychometric research that wasn't relatively easily explained as culturally tied. > but as long as they continue to maintain a relatively consistent relationship to observable reality The point is that neither this study nor the homosexuality study have a relatively consistent relationship to observable reality. Your priors on us being able to predict, independent of social conditioning, some arbitrary social attribute based on someone's face should be very, very low. And "predicting" some arbitrary social attribute based on social conditioning is just encoding social bias into the model, which is bad. > and that if we had continually given higher credence to feelings the Enlightenment and most scientific progress we've had would have been far slower if it had happened at all. You mean like all the scientific progress that came out of phrenological research?
- NotSammyHagar 6y ago
  They are pointing out that facial recognition has a lot of the problems that these earlier schemes had in that they were another way to disenfranchise and cause problems for minorities. With FR the immediate issue is making many false matches, leading to yet another thing causing over-policing with darker skinned people.
  - Udik 6y ago
    > With FR the immediate issue is making many false matches I don't get it though. False matches seem to be a great thing: it means that facial recognition does not really work well and can't be used to tell who is where. Do you prefer FR that never makes mistakes?
    - d_watt 6y ago
      The concern here is that like a dog sniffing a car and barking, the false matches give erroneous probable cause. Combo this with systems that are more likely to give false negatives to minorities, and you've created a blackhole of accountability for authoritarian harassment.
      - Udik 6y ago
        But then your problem is with authoritarian harassment, not with facial recognition. If you're fine with the fact that the police can stop and harass random people based on indicators that have a high rate of false positives, then facial recognition is not the first of your issues.
        Infinitesimus 6y ago
        No it isn't the first of our issues. If there is a fire, you don't pour kerosene on it while trying to put it out, right? Adding more tools we know will be abused at a time like this seems like adding fuel to fire and removing accountability from individual actions. You might have a better chance fighting a case where you are stopped for no reason than fighting a case where your car is pulled over because the FR system mistook your blurry face for that of someone else due to poor lighting.
rexreed 6y ago
It's too late.
- core-questions 6y ago
  It was too late even before facial recognition technology existed, because it's not like you can't apply modern tech to older video footage.
7ArcticSealz 6y ago
...I will have to hide my elongated skull now...
JoshuaDavid 6y ago
I am not sure what "defunding" would accomplish at this point. At this point, the research has been done, and the products are available for astonishingly low prices. We don't stop oil spills by defunding offshore oil wells that have already been dug, and attempting to do so is likely to result in more oil spills as the money for maintenance is cut. Similarly, I'd expect that if budget for facial recognition is slashed, the parts that will be cut are oversight and training, and so now instead of just dealing with questionable technology, you're dealing with questionable technology used by people who have no idea what they're doing. Also, as long as public spaces are under constant video surveillance, stopping facial recognition now only solves the problem temporarily. I think at a bare minimum, we need standards for when this evidence should be admissible in court (at current tech levels, probably approximately never) and when it is acceptable to use it in searches. The technical ship has sailed, so any fix is going to have to be legislative at this point.
- DyslexicAtheist 6y ago
  "> Amazon, Microsoft, and Google have continued efforts to ensure federal regulation that offers a stable and profitable market in which facial-recognition technology is, in fact, used by law enforcement, in direct opposition to the movement the companies claim to support." if these Tech companies were serious about fighting inequality it would be more effective to start banning racist AI/facial recognition instead of scrubbing Technology language from words like "blacklist", and "master/slave".
m0zg 6y ago
Sorry, I'd much rather defund The Atlantic. Facial recognition at least has valid uses beyond surveillance. The Atlantic has no known good uses that I can discern.

23 points by sneeze-slayer 6y ago

> Rooted in discredited pseudoscience and racist eugenics theories that claim to use facial structure and head shape to assess mental capacity and character, automated facial-recognition software uses artificial intelligence, machine learning, and other forms of modern computing to capture the details of people’s faces and compare that information to existing photo databases with the goal of identifying, verifying, categorizing, and locating people. I'm sorry, what? Does this person think that phrenology / physiognomy, two old pseudosciences that have been discredited for a hundred years or more, are actually at play within ML systems? I'm totally willing to believe that ML facial recognition systems insufficiently trained on a wide enough set of faces will mistake one person for another. Sure. But to pretend that the system is based on eugenics belies a critical lack of understanding of what these things actually do, and ascribes agency and racial animus to a computer program. Pretty clear to me that this shows the author doesn't really know anything about how these things work. The reason to not want facial recognition in public spaces is the same as to not want mass surveillance: a reasonable expectation of privacy by citizens. Of course, if also want no police in these areas at the same time, one should not be surprised if eventually they go from peaceable public squares, to a haven for petty crime, to eventually a fearful place people avoid.

joshuamorton 6y ago
> I'm sorry, what? Does this person think that phrenology / physiognomy, two old pseudosciences that have been discredited for a hundred years or more, are actually at play within ML systems? https://callingbullshit.org/case_studies/case_study_criminal_machine_learning.html https://callingbullshit.org/case_studies/case_study_criminal... This shouldn't be a problem, because as you note phrenology is pseudoscience that's been discredited for over a century. And yet. To the broader point, I'd argue that in generally any attempt to predict criminality from facial structure/face picture is phrenological in nature. And people who do know what's at play with ML systems do agree with this take: https://twitter.com/ylecun/status/1276147230295166984 https://twitter.com/ylecun/status/1276147230295166984
- core-questions 6y ago
  > And yet. Well, what's happening there is not a study of phrenology at all (which posited specific regions of facial/skull structure being indicators). It's actually a very interesting thing to look at. There was a previous one that was reporting some degree of success in determining whether someone was homosexual via ML. Here's the thing: if this turns out to have actual predictive power, then it's a subject worthy of scientific study, whether you like the outcomes and conclusions or not. Plenty of other worthy areas of endeavour (e.g. psychometric IQ research) have also revealed uncomfortable truths. If instead these things turn out to not have any legitimate research value (i.e. can't make predictions that can be experimentally verified), then we can stop looking at them, but as long as they continue to maintain a relatively consistent relationship to observable reality, they're as worthy a form of science as any other anthropological research is. We have a choice either face this head on, include it in policies, and build our sociology around the truth, or we can put our heads in the sand to make people feel better. I for one believe that truth is far more important than feelings, and that if we had continually given higher credence to feelings the Enlightenment and most scientific progress we've had would have been far slower if it had happened at all.
  - joshuamorton 6y ago
    > one that was reporting some degree of success in determining whether someone was homosexual via ML Yes, which was also phrenological in nature. > one that was reporting some degree of success in determining whether someone was homosexual via ML I've yet to see any "uncomfortable truth" from psychometric research that wasn't relatively easily explained as culturally tied. > but as long as they continue to maintain a relatively consistent relationship to observable reality The point is that neither this study nor the homosexuality study have a relatively consistent relationship to observable reality. Your priors on us being able to predict, independent of social conditioning, some arbitrary social attribute based on someone's face should be very, very low. And "predicting" some arbitrary social attribute based on social conditioning is just encoding social bias into the model, which is bad. > and that if we had continually given higher credence to feelings the Enlightenment and most scientific progress we've had would have been far slower if it had happened at all. You mean like all the scientific progress that came out of phrenological research?
NotSammyHagar 6y ago
They are pointing out that facial recognition has a lot of the problems that these earlier schemes had in that they were another way to disenfranchise and cause problems for minorities. With FR the immediate issue is making many false matches, leading to yet another thing causing over-policing with darker skinned people.
- Udik 6y ago
  > With FR the immediate issue is making many false matches I don't get it though. False matches seem to be a great thing: it means that facial recognition does not really work well and can't be used to tell who is where. Do you prefer FR that never makes mistakes?
  - d_watt 6y ago
    The concern here is that like a dog sniffing a car and barking, the false matches give erroneous probable cause. Combo this with systems that are more likely to give false negatives to minorities, and you've created a blackhole of accountability for authoritarian harassment.
    - Udik 6y ago
      But then your problem is with authoritarian harassment, not with facial recognition. If you're fine with the fact that the police can stop and harass random people based on indicators that have a high rate of false positives, then facial recognition is not the first of your issues.
      - Infinitesimus 6y ago
        No it isn't the first of our issues. If there is a fire, you don't pour kerosene on it while trying to put it out, right? Adding more tools we know will be abused at a time like this seems like adding fuel to fire and removing accountability from individual actions. You might have a better chance fighting a case where you are stopped for no reason than fighting a case where your car is pulled over because the FR system mistook your blurry face for that of someone else due to poor lighting.

rexreed 6y ago

It's too late.

core-questions 6y ago
It was too late even before facial recognition technology existed, because it's not like you can't apply modern tech to older video footage.

7ArcticSealz 6y ago

...I will have to hide my elongated skull now...

JoshuaDavid 6y ago

I am not sure what "defunding" would accomplish at this point. At this point, the research has been done, and the products are available for astonishingly low prices. We don't stop oil spills by defunding offshore oil wells that have already been dug, and attempting to do so is likely to result in more oil spills as the money for maintenance is cut. Similarly, I'd expect that if budget for facial recognition is slashed, the parts that will be cut are oversight and training, and so now instead of just dealing with questionable technology, you're dealing with questionable technology used by people who have no idea what they're doing. Also, as long as public spaces are under constant video surveillance, stopping facial recognition now only solves the problem temporarily. I think at a bare minimum, we need standards for when this evidence should be admissible in court (at current tech levels, probably approximately never) and when it is acceptable to use it in searches. The technical ship has sailed, so any fix is going to have to be legislative at this point.

DyslexicAtheist 6y ago
"> Amazon, Microsoft, and Google have continued efforts to ensure federal regulation that offers a stable and profitable market in which facial-recognition technology is, in fact, used by law enforcement, in direct opposition to the movement the companies claim to support." if these Tech companies were serious about fighting inequality it would be more effective to start banning racist AI/facial recognition instead of scrubbing Technology language from words like "blacklist", and "master/slave".

m0zg 6y ago

Sorry, I'd much rather defund The Atlantic. Facial recognition at least has valid uses beyond surveillance. The Atlantic has no known good uses that I can discern.