5 ms·

Surprised they weren't more careful with this after Google had the same problem a few years ago. https://www.theverge.com/2018/1/12/16882408/google-racist-gori

Surprised they weren't more careful with this after Google had the same problem a few years ago. https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai https://www.theverge.com/2018/1/12/16882408/google-racist-go...

bryan0 5y ago
Yeah you’d think think they be extra careful on their model which classifies videos of people before releasing it.
- dralley 5y ago
  Why would you think the "move fast and break things" company would be more careful?
hn_throwaway_99 5y ago
OK, I'll try to tread carefully here, but I think it's really helpful to separate the technical issues from the societal issues. In the Google photo case, I've seen the picture in question, and it's not difficult an all to see how a rudimentary statistical algorithm would mistake the photo for a gorilla. No, no human would make this mistake, but due to the lighting in the photo and the woman's hair it's really not hard to see how that mistake would be made, similar to how Tesla's AI can mistake the sun, low on the horizon, as a yellow traffic light. The algorithm was not "racist", it just didn't have the social context that describing black people as apes and monkeys has a long racist history. And that is really the fundamental problem with most AI these days - it can get very good at statistical inference, but it doesn't have the background knowledge and logic to be able to "think" in the same way humans do. In a similar vein, I recall someone lamenting a couple years ago how Google's street directions would never mispronounce "Malcolm X Boulevard" as "Malcolm 10 Boulevard" if they had any black programmers. Again, given 99% of the time when you see "X" in an address you'd pronounce it as 10, it's not hard to see how this could happen. The problem is that some errors are much more offensive than others, and AI can't really reason about that.
- ant6n 5y ago
  The problem is the training set, which apparently skews too much towards white ppl, and doesn't include checks for the primates/non-white humans differentiation. One could argue this is due to white people building things for white ppl, which does indeed point towards systemic racism in the overall development process.
  - fallingknife 5y ago
    But the problem is that there isn't any primates/non-white humans differentiation because humans are primates. I doubt anybody smart enough to be an AI engineer of any race would care if they were called a primate, because they are.
    - hn_throwaway_99 5y ago
      I feel like this is a very "engineer" response that completely ignores how society in general would view this. Yes, humans are primates. But this wasn't a discussion about taxonomy in biology class. There are lots of cases where primates is used in everyday language where it is clearly intended to refer solely to monkeys and apes. When I go to the "primate house" at the zoo I'm not going to expect to see naked humans behind the glass.
      - fallingknife 5y ago
        But an AI image recognition system is a very "engineer" kind of thing.
        adventured 5y ago
        And in cases where that engineer thing is used on people - it matters when you label a black person as a primate on a live social media network, since it's understood that would be likely to cause significant offense.
        tartoran 5y ago
amelius 5y ago
> Surprised they weren't more careful with this They were careful, otherwise they wouldn't have labeled it "primates", but e.g. "monkeys" instead.
ksaj 5y ago
For Asians, there is a history of cameras saying "Someone Blinked!" I think it is, or at least should be, part of the conversation. My partner is Asian, and we had a camera with this problem. We thought it was funny, but it demonstrates weaknesses in computer image processing vs. human visual processing that probably has nothing to do with racism.

bryan0 5y ago

Yeah you’d think think they be extra careful on their model which classifies videos of people before releasing it.

dralley 5y ago
Why would you think the "move fast and break things" company would be more careful?

hn_throwaway_99 5y ago

OK, I'll try to tread carefully here, but I think it's really helpful to separate the technical issues from the societal issues. In the Google photo case, I've seen the picture in question, and it's not difficult an all to see how a rudimentary statistical algorithm would mistake the photo for a gorilla. No, no human would make this mistake, but due to the lighting in the photo and the woman's hair it's really not hard to see how that mistake would be made, similar to how Tesla's AI can mistake the sun, low on the horizon, as a yellow traffic light. The algorithm was not "racist", it just didn't have the social context that describing black people as apes and monkeys has a long racist history. And that is really the fundamental problem with most AI these days - it can get very good at statistical inference, but it doesn't have the background knowledge and logic to be able to "think" in the same way humans do. In a similar vein, I recall someone lamenting a couple years ago how Google's street directions would never mispronounce "Malcolm X Boulevard" as "Malcolm 10 Boulevard" if they had any black programmers. Again, given 99% of the time when you see "X" in an address you'd pronounce it as 10, it's not hard to see how this could happen. The problem is that some errors are much more offensive than others, and AI can't really reason about that.

ant6n 5y ago
The problem is the training set, which apparently skews too much towards white ppl, and doesn't include checks for the primates/non-white humans differentiation. One could argue this is due to white people building things for white ppl, which does indeed point towards systemic racism in the overall development process.
- fallingknife 5y ago
  But the problem is that there isn't any primates/non-white humans differentiation because humans are primates. I doubt anybody smart enough to be an AI engineer of any race would care if they were called a primate, because they are.
  - hn_throwaway_99 5y ago
    I feel like this is a very "engineer" response that completely ignores how society in general would view this. Yes, humans are primates. But this wasn't a discussion about taxonomy in biology class. There are lots of cases where primates is used in everyday language where it is clearly intended to refer solely to monkeys and apes. When I go to the "primate house" at the zoo I'm not going to expect to see naked humans behind the glass.
    - fallingknife 5y ago
      But an AI image recognition system is a very "engineer" kind of thing.
      - adventured 5y ago
        And in cases where that engineer thing is used on people - it matters when you label a black person as a primate on a live social media network, since it's understood that would be likely to cause significant offense.
      - tartoran 5y ago

amelius 5y ago

> Surprised they weren't more careful with this They were careful, otherwise they wouldn't have labeled it "primates", but e.g. "monkeys" instead.

ksaj 5y ago

For Asians, there is a history of cameras saying "Someone Blinked!" I think it is, or at least should be, part of the conversation. My partner is Asian, and we had a camera with this problem. We thought it was funny, but it demonstrates weaknesses in computer image processing vs. human visual processing that probably has nothing to do with racism.