7 ms·

Good. This kind of software is a mirror to the data fed into it. People on the edges of society (that the people making the mirror might either intentionally or

by xena 7y ago

Good. This kind of software is a mirror to the data fed into it. People on the edges of society (that the people making the mirror might either intentionally or accidentally ignore) will get fucked over tremendously by this technology. We should put a complete ban on it. Even if it means gimping my photo organization programs.

diafygi 7y ago
Just so I have good arguments when talking to someone about this specific point, can you or someone else please go into more detail on why this will happen?
- donohoe 7y ago
  Not specific to facial recognition but covers it and the deeper issues. It is a thorough yet accessible primer on the many ways bias can creep into these systems: https://parametric.press/issue-01/the-myth-of-the-impartial-machine/ https://parametric.press/issue-01/the-myth-of-the-impartial-...
  - masklinn 7y ago
    As @pinboard noted back in the dark ages of 2016, "machine learning is money laundering for bias".
- paganel 7y ago
  Not the OP by technology like this will explicitly make racial profiling legal again, because the power that be will hide behind the decisions taken by algorithms: “The computer told us to stop and search this black person’s car, it wasn’t us, the computer is smart, we were just following the computer’s orders”.
  - bumby 7y ago
    I heard this referred to recently as the "automation fallacy"
  - xena 7y ago
    As a transgender woman, this is my fear precisely.
  - chr1 7y ago
    That's why there is https://en.wikipedia.org/wiki/Right_to_explanation https://en.wikipedia.org/wiki/Right_to_explanation
- trevyn 7y ago
  Along the lines of “everyone is breaking some law”, there could be a shift from “technically illegal but generally harmless and tolerated” to “illegal and mindlessly enforced by automated technology”. An example being China’s automated jaywalking-shamer.
  - mc32 7y ago
    The only “good” outcome of this is we get to know how pervasive surveillance affects behavior and whether the benefits outweigh the detriments. In other words does it result in a better or worse society overall. Even if it’s better overall I’d it worth the negative effects on affected populations? It’s a living laboratory.
    - homonculus1 7y ago
      The benefits would have to be impossibly valuable to outweigh the constant burden and psychological damage of living in a totalitarian panopticon. Given that society functions without it and is constantly improving anyway--crime rates, poverty, mortality continue to drop from socioeconomic and technological development without the need for the intervention of pervasive surveillance--I personally literally cannot conceive of a benefit that would actually be worth it.
      - mc32 7y ago
        True, I do not see it turning out well. But it would be a laboratory into behavioral control. Something like this would eliminate needs for safe spaces, anti/hate speech, etc. because this would regulate all this anti social behavior in the first place.
- bryanrasmussen 7y ago
  There have been numerous issues where corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces and therefore be shut out of whatever the facial recognition is supposed to implement. For example facial recognition at a high priced building, suddenly non-white people can't get in because you did not show a real face at the scanner. Sorry, it's a bug not a feature. The same problem would of course apply to anyone with facial deformities etc. The problem would be essentially of two forms - Automatic exclusion from a group by facial feature, or automatic inclusion in a group by facial feature. Think of the automatic inclusion as advanced phrenology or other sort of woo, in this scenario company X sells the idea that they will have facial recognition of 'criminal types' based on whatever research they can pull out that criminals often correspond in appearance. Then they fill that corpus up with criminals which then is a nice self-selecting system of find the black/hispanic person and deny them the job, loan etc. because of likelihood for crime past the threshold set. the reason why this would happen is the same reason why banks used to deny loans to people in certain zip codes, because by doing so they could be racially profiling and excluding but pretend not to do so.
  - mc32 7y ago
    >”corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces..” I’m not sure this is s good argument against the technology because there is a solution to this objection: they’ll improve it and ensure every face type is comparably recognized. It should be rejected on more basic principles.
    - rmrfrmrf 7y ago
      It's not as simple as "just make sure it works on everyone". Even if it were physically possible to capture the face of all 7 billion people on Earth, you'd start running into issues of false positives/negatives, overfitting, etc., and furthermore it would basically require every person to be periodically scanned from birth until death to ensure that the algorithm can recognize new faces.
- lazyvoice 7y ago
  For example, let's assume you live in Brunei where homosexuality is a capital crime. Facial-recognition (especially in concert with other pattern-recognition software) drastically reduces the man-power necessary to identify individuals for enforcement. If you're talking to someone who's more conservatively inclined I'd talk about how Iceland has managed to basically eliminate Down Syndrome. Prenatal pattern recognition has successfully convinced nearly all Icelandic parents who have a pregnancy where the child may have Down Syndrome to abort. If your listener doesn't care about gay rights AND thinks any fetus with a higher likelihood of having Down Syndrome should be aborted, then they're probably exactly the sort of people working on or hoping to use these tools.
  - mnw21cam 7y ago
    Okay, I see your Down Syndrome point, and raise you Autism/Asperger Syndrome. Alright, there isn't currently a test that would enable people with Autism/Asperger Syndrome to be systematically aborted, but it isn't so hard to imagine a future where it would be possible. Given the high proportion of people in this forum with these conditions, how do we feel about this?
    - phissk 7y ago
      Please note that you're not aborting 'people' ... just a thing that might become a person. You're giving (actual) people the opportunity to choose to have healthier families. - Note that I say 'healthier families' specifically because willfully passing on genetic features that disadvantage your children burdens not just them but your entire family unit.
      - mistersquid 7y ago
        The problem lies within the generalizations presented by the words “healthier” and “burdens”. Many families with Downs children describe how much joy and love their Downs children bring to the family, how the family members learned to cherish what’s important in life, etc. These families compared to many families with non-Downs children are quite healthy and arguably not burdened though they experience different challenges than other families. EDIT: fix quote; change last preposition in second sentence.
dsfyu404ed 7y ago
I fully agree with your sentiment but I don't agree with a ban. Prohibition doesn't work. You should be able to use it but the legal risks and liability you open yourself up to by using it should be strong enough to deter its use except when there truly is no substitute. Asbestos comes to mind as an example. People (often on HN) often run in circles and scream about the sky falling because "OMG it's not banned" but the reality is that the court of public opinion has basically ruled it is not ok to use. That's how I want to see facial recognition tech and other tech that enables corporations and governments to stalk people at scale treated
- rmrfrmrf 7y ago
  > Prohibition doesn't work You're appropriating sociological behaviors of the individual to maxims at the corporate level, which is ridiculous. Corporations need to be regulated, not coddled in some kind of convoluted, scaled-up, harm-reduction policy.
  - dsfyu404ed 7y ago
    Public opinion is what ultimately is the determining factor here. If the public does not strongly want to punish companies for doing something then the government will not be successful at punishing companies for doing something. In a democracy government follows public opinion. In the absence of strong public opinion you get corporate interests shaping what government does. Yeah, sure, ban it. But without social consensus backing up those laws it will have no teeth in practice.
    - ghostDancer 7y ago
      Amen to that. And public opinion is easily manipulated. Create fear and people will give up their rights , if someone resists accuse them of being anti-freedom, anti-patriotic, against the free market, communist/socialist, helping terrorists,etc. Use the media which is owned by the same people politicians protect and in no time most of the population will comply
- xena 7y ago
stared 7y ago
As a person with prosopagnosia (vide http://web.archive.org/web/20080704154655/http://www.prosopagnosia.com/main/stones/index.asp http://web.archive.org/web/20080704154655/http://www.prosopa...), I would love to have personal software for recognizing people. At the same time, I am aware of the dangers of people doing it for commercial purposes (on so many levels!).
- jasonlotito 7y ago
  It's things like this that always make me think twice about banning things. Many people look at something as useless, wasteful, or bad, but to other people, it really does help people with certain conditions. So when you outright declare something is dumb or should be banned, I generally find these people to generally be ignorant about all the technology does, and it's just a knee-jerk reaction. This is not to say they don't have good reasons for not liking a technology. Rather, that tend to not know all sides for something.
- JamesAdir 7y ago
  check out this: https://www.kickstarter.com/projects/1070992105/orcam-myme1st-of-its-kind-wearable-ai-2nd-brain-3r https://www.kickstarter.com/projects/1070992105/orcam-myme1s...
  - stared 7y ago
    Interesting. Does it work? (Any reviews, since it's past its release date?)
toephu2 7y ago
You have never been a victim of a crime have you? I'm pretty sure if you were, and the crime was caught on camera, your position on facial-recognition software would change.
- ashelmire 7y ago
  Or you'll just get harassed more as a black person[1]. Funny enough (actually the opposite of funny), the systems used by law enforcement are designed to create more false positives, which then leaves it up to law enforcement to decide which "leads" to follow up on. This will certainly be used to validate abusive behavior. Additionally, facial recognition is used in some countries right now to silence political dissidents[2]. If you think that can't or won't happen in the US, I'd suggest you look at how activists have been treated in the past (beaten, arrested, and sometimes killed). 1. https://www.nytimes.com/2018/02/09/technology/facial-recognition-race-artificial-intelligence.html https://www.nytimes.com/2018/02/09/technology/facial-recogni... 2. https://www.eff.org/pages/face-recognition https://www.eff.org/pages/face-recognition
- Barrin92 7y ago
  >I'm pretty sure if you were, and the crime was caught on camera, your position on facial-recognition software would change. yes, but likely in the wrong direction because I'd be angry and personally insecure. Decisions of liberty, privacy and law shouldn't be based on personal experiences or trauma. I find this line of argument weird. Do you really think a personal decision about long term social issues is better than an impersonal one?
mr_toad 7y ago
> We should put a complete ban on it. Both my phone and my PS4 use facial recognition for login. I don’t want that banned. My camera can be set up to take a photo when it detects someone in the frame. Very useful for selfies. Are you going to ban that too. Are you going to ban Tensorflow. Keras? SciKit-learn? If I hand code my own neural net, are you going to ban that as well?
- vangelis 7y ago
  I disagree with a complete ban, but we could ban wide-scale implementation of this. You are not a municipality.

by xena 7y ago

diafygi 7y ago

Just so I have good arguments when talking to someone about this specific point, can you or someone else please go into more detail on why this will happen?

donohoe 7y ago
Not specific to facial recognition but covers it and the deeper issues. It is a thorough yet accessible primer on the many ways bias can creep into these systems: https://parametric.press/issue-01/the-myth-of-the-impartial-machine/ https://parametric.press/issue-01/the-myth-of-the-impartial-...
- masklinn 7y ago
  As @pinboard noted back in the dark ages of 2016, "machine learning is money laundering for bias".
paganel 7y ago
Not the OP by technology like this will explicitly make racial profiling legal again, because the power that be will hide behind the decisions taken by algorithms: “The computer told us to stop and search this black person’s car, it wasn’t us, the computer is smart, we were just following the computer’s orders”.
- bumby 7y ago
  I heard this referred to recently as the "automation fallacy"
- xena 7y ago
  As a transgender woman, this is my fear precisely.
- chr1 7y ago
  That's why there is https://en.wikipedia.org/wiki/Right_to_explanation https://en.wikipedia.org/wiki/Right_to_explanation
trevyn 7y ago
Along the lines of “everyone is breaking some law”, there could be a shift from “technically illegal but generally harmless and tolerated” to “illegal and mindlessly enforced by automated technology”. An example being China’s automated jaywalking-shamer.
- mc32 7y ago
  The only “good” outcome of this is we get to know how pervasive surveillance affects behavior and whether the benefits outweigh the detriments. In other words does it result in a better or worse society overall. Even if it’s better overall I’d it worth the negative effects on affected populations? It’s a living laboratory.
  - homonculus1 7y ago
    The benefits would have to be impossibly valuable to outweigh the constant burden and psychological damage of living in a totalitarian panopticon. Given that society functions without it and is constantly improving anyway--crime rates, poverty, mortality continue to drop from socioeconomic and technological development without the need for the intervention of pervasive surveillance--I personally literally cannot conceive of a benefit that would actually be worth it.
    - mc32 7y ago
      True, I do not see it turning out well. But it would be a laboratory into behavioral control. Something like this would eliminate needs for safe spaces, anti/hate speech, etc. because this would regulate all this anti social behavior in the first place.
bryanrasmussen 7y ago
There have been numerous issues where corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces and therefore be shut out of whatever the facial recognition is supposed to implement. For example facial recognition at a high priced building, suddenly non-white people can't get in because you did not show a real face at the scanner. Sorry, it's a bug not a feature. The same problem would of course apply to anyone with facial deformities etc. The problem would be essentially of two forms - Automatic exclusion from a group by facial feature, or automatic inclusion in a group by facial feature. Think of the automatic inclusion as advanced phrenology or other sort of woo, in this scenario company X sells the idea that they will have facial recognition of 'criminal types' based on whatever research they can pull out that criminals often correspond in appearance. Then they fill that corpus up with criminals which then is a nice self-selecting system of find the black/hispanic person and deny them the job, loan etc. because of likelihood for crime past the threshold set. the reason why this would happen is the same reason why banks used to deny loans to people in certain zip codes, because by doing so they could be racially profiling and excluding but pretend not to do so.
- mc32 7y ago
  >”corpora of peoples faces have been insufficiently stocked with a particular racial group, in such cases people's faces might not register as faces..” I’m not sure this is s good argument against the technology because there is a solution to this objection: they’ll improve it and ensure every face type is comparably recognized. It should be rejected on more basic principles.
  - rmrfrmrf 7y ago
    It's not as simple as "just make sure it works on everyone". Even if it were physically possible to capture the face of all 7 billion people on Earth, you'd start running into issues of false positives/negatives, overfitting, etc., and furthermore it would basically require every person to be periodically scanned from birth until death to ensure that the algorithm can recognize new faces.
lazyvoice 7y ago
For example, let's assume you live in Brunei where homosexuality is a capital crime. Facial-recognition (especially in concert with other pattern-recognition software) drastically reduces the man-power necessary to identify individuals for enforcement. If you're talking to someone who's more conservatively inclined I'd talk about how Iceland has managed to basically eliminate Down Syndrome. Prenatal pattern recognition has successfully convinced nearly all Icelandic parents who have a pregnancy where the child may have Down Syndrome to abort. If your listener doesn't care about gay rights AND thinks any fetus with a higher likelihood of having Down Syndrome should be aborted, then they're probably exactly the sort of people working on or hoping to use these tools.
- mnw21cam 7y ago
  Okay, I see your Down Syndrome point, and raise you Autism/Asperger Syndrome. Alright, there isn't currently a test that would enable people with Autism/Asperger Syndrome to be systematically aborted, but it isn't so hard to imagine a future where it would be possible. Given the high proportion of people in this forum with these conditions, how do we feel about this?
  - phissk 7y ago
    Please note that you're not aborting 'people' ... just a thing that might become a person. You're giving (actual) people the opportunity to choose to have healthier families. - Note that I say 'healthier families' specifically because willfully passing on genetic features that disadvantage your children burdens not just them but your entire family unit.
    - mistersquid 7y ago
      The problem lies within the generalizations presented by the words “healthier” and “burdens”. Many families with Downs children describe how much joy and love their Downs children bring to the family, how the family members learned to cherish what’s important in life, etc. These families compared to many families with non-Downs children are quite healthy and arguably not burdened though they experience different challenges than other families. EDIT: fix quote; change last preposition in second sentence.

dsfyu404ed 7y ago

I fully agree with your sentiment but I don't agree with a ban. Prohibition doesn't work. You should be able to use it but the legal risks and liability you open yourself up to by using it should be strong enough to deter its use except when there truly is no substitute. Asbestos comes to mind as an example. People (often on HN) often run in circles and scream about the sky falling because "OMG it's not banned" but the reality is that the court of public opinion has basically ruled it is not ok to use. That's how I want to see facial recognition tech and other tech that enables corporations and governments to stalk people at scale treated

rmrfrmrf 7y ago
> Prohibition doesn't work You're appropriating sociological behaviors of the individual to maxims at the corporate level, which is ridiculous. Corporations need to be regulated, not coddled in some kind of convoluted, scaled-up, harm-reduction policy.
- dsfyu404ed 7y ago
  Public opinion is what ultimately is the determining factor here. If the public does not strongly want to punish companies for doing something then the government will not be successful at punishing companies for doing something. In a democracy government follows public opinion. In the absence of strong public opinion you get corporate interests shaping what government does. Yeah, sure, ban it. But without social consensus backing up those laws it will have no teeth in practice.
  - ghostDancer 7y ago
    Amen to that. And public opinion is easily manipulated. Create fear and people will give up their rights , if someone resists accuse them of being anti-freedom, anti-patriotic, against the free market, communist/socialist, helping terrorists,etc. Use the media which is owned by the same people politicians protect and in no time most of the population will comply
xena 7y ago

stared 7y ago

As a person with prosopagnosia (vide http://web.archive.org/web/20080704154655/http://www.prosopagnosia.com/main/stones/index.asp http://web.archive.org/web/20080704154655/http://www.prosopa...), I would love to have personal software for recognizing people. At the same time, I am aware of the dangers of people doing it for commercial purposes (on so many levels!).

jasonlotito 7y ago
It's things like this that always make me think twice about banning things. Many people look at something as useless, wasteful, or bad, but to other people, it really does help people with certain conditions. So when you outright declare something is dumb or should be banned, I generally find these people to generally be ignorant about all the technology does, and it's just a knee-jerk reaction. This is not to say they don't have good reasons for not liking a technology. Rather, that tend to not know all sides for something.
JamesAdir 7y ago
check out this: https://www.kickstarter.com/projects/1070992105/orcam-myme1st-of-its-kind-wearable-ai-2nd-brain-3r https://www.kickstarter.com/projects/1070992105/orcam-myme1s...
- stared 7y ago
  Interesting. Does it work? (Any reviews, since it's past its release date?)

toephu2 7y ago

You have never been a victim of a crime have you? I'm pretty sure if you were, and the crime was caught on camera, your position on facial-recognition software would change.

ashelmire 7y ago
Or you'll just get harassed more as a black person[1]. Funny enough (actually the opposite of funny), the systems used by law enforcement are designed to create more false positives, which then leaves it up to law enforcement to decide which "leads" to follow up on. This will certainly be used to validate abusive behavior. Additionally, facial recognition is used in some countries right now to silence political dissidents[2]. If you think that can't or won't happen in the US, I'd suggest you look at how activists have been treated in the past (beaten, arrested, and sometimes killed). 1. https://www.nytimes.com/2018/02/09/technology/facial-recognition-race-artificial-intelligence.html https://www.nytimes.com/2018/02/09/technology/facial-recogni... 2. https://www.eff.org/pages/face-recognition https://www.eff.org/pages/face-recognition
Barrin92 7y ago
>I'm pretty sure if you were, and the crime was caught on camera, your position on facial-recognition software would change. yes, but likely in the wrong direction because I'd be angry and personally insecure. Decisions of liberty, privacy and law shouldn't be based on personal experiences or trauma. I find this line of argument weird. Do you really think a personal decision about long term social issues is better than an impersonal one?

mr_toad 7y ago

> We should put a complete ban on it. Both my phone and my PS4 use facial recognition for login. I don’t want that banned. My camera can be set up to take a photo when it detects someone in the frame. Very useful for selfies. Are you going to ban that too. Are you going to ban Tensorflow. Keras? SciKit-learn? If I hand code my own neural net, are you going to ban that as well?

vangelis 7y ago
I disagree with a complete ban, but we could ban wide-scale implementation of this. You are not a municipality.