4 ms·

The space for potential edge cases are so large it will be wack mole. Inference tasks isn’t like software testing where the states are well defined.

dontreact 5y ago
Yeah, but it shows some real lack of care to fall into the same edge case that Google was widely criticized for 3 years ago.
- bryan0 5y ago
  I don’t think this is an edge case. We are asking AI to classify images and videos of people where the results can be disastrous.
daenz 5y ago
A large and complex task is well suited for the talented people behind Facebook. But even a list of basic searches that exercised the engine would have caught this. No excuses imo, particularly since it happened to Google[0] not that long ago. Was nobody paying attention? 0. https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai https://www.theverge.com/2018/1/12/16882408/google-racist-go...
- ta8902 5y ago
  Maybe they thought it would be racist if they explicitly devised rules preventing black people being identified as primates.
stathibus 5y ago
Another way to put this is: nobody working in this field has control over what they are building, and can't make any promises about how it will behave. It's like launching a rocket ship that you can't test in a physical simulation first. Something is going to go wrong that can't be predicted because we lack the understanding, but this is seen as an acceptable risk.
- toxik 5y ago
  Which is inherent to the problem. The domain is the space of _all possible natural images_, it’s so big, it’s ridiculous. Fundamentally, these techniques do not analyze images like humans do, but are rather trained to pick out any salient signal it can latch on to. That seems to be “primates are mostly like humans but darker”, which is superficially true but a pretty weak definition as it includes dark skinned humans.
  - wizzwizz4 5y ago
    It's probably because humans are primates – but the AI systems often have to treat “human” as a completely separate category as “primate”, so they have to draw weird, complex boundaries around “primate” (actually “all non-human primates”). When the “primate” classification is stronger than the “human” classification, the system says “primate” rather than “human”, and if it's predominantly been trained on “pictures of white Americans are not pictures of primates”, its “primate” definition might not be skewed to miss everyone else. I expect you'd get better results if you allowed the system to call humans “primates”, then accept “human primate” as “human” when parsing the output. (That is, leave the “is_primate” output line floating while training on pictures of humans.) I don't know whether that would work, though.
    - toxik 5y ago
      These models typically don’t have hierarchical labels like that, and they apply a softmax to their output - which means /one/ label will be considered correct. (A softmax means taking the exp of your predicted scores, then divide by the sum.)
      - wizzwizz4 5y ago
        I know – but there's no technical reason they shouldn't have more complex relationships between labels (assuming you can train that, which I don't know). If we can't get better training data, at least trying to fix the problem at the algorithms (instead of slapping crude filters on the end of them) would be nice.

dontreact 5y ago

Yeah, but it shows some real lack of care to fall into the same edge case that Google was widely criticized for 3 years ago.

bryan0 5y ago
I don’t think this is an edge case. We are asking AI to classify images and videos of people where the results can be disastrous.

daenz 5y ago

A large and complex task is well suited for the talented people behind Facebook. But even a list of basic searches that exercised the engine would have caught this. No excuses imo, particularly since it happened to Google[0] not that long ago. Was nobody paying attention? 0. https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai https://www.theverge.com/2018/1/12/16882408/google-racist-go...

ta8902 5y ago
Maybe they thought it would be racist if they explicitly devised rules preventing black people being identified as primates.

stathibus 5y ago

Another way to put this is: nobody working in this field has control over what they are building, and can't make any promises about how it will behave. It's like launching a rocket ship that you can't test in a physical simulation first. Something is going to go wrong that can't be predicted because we lack the understanding, but this is seen as an acceptable risk.

toxik 5y ago
Which is inherent to the problem. The domain is the space of _all possible natural images_, it’s so big, it’s ridiculous. Fundamentally, these techniques do not analyze images like humans do, but are rather trained to pick out any salient signal it can latch on to. That seems to be “primates are mostly like humans but darker”, which is superficially true but a pretty weak definition as it includes dark skinned humans.
- wizzwizz4 5y ago
  It's probably because humans are primates – but the AI systems often have to treat “human” as a completely separate category as “primate”, so they have to draw weird, complex boundaries around “primate” (actually “all non-human primates”). When the “primate” classification is stronger than the “human” classification, the system says “primate” rather than “human”, and if it's predominantly been trained on “pictures of white Americans are not pictures of primates”, its “primate” definition might not be skewed to miss everyone else. I expect you'd get better results if you allowed the system to call humans “primates”, then accept “human primate” as “human” when parsing the output. (That is, leave the “is_primate” output line floating while training on pictures of humans.) I don't know whether that would work, though.
  - toxik 5y ago
    These models typically don’t have hierarchical labels like that, and they apply a softmax to their output - which means /one/ label will be considered correct. (A softmax means taking the exp of your predicted scores, then divide by the sum.)
    - wizzwizz4 5y ago
      I know – but there's no technical reason they shouldn't have more complex relationships between labels (assuming you can train that, which I don't know). If we can't get better training data, at least trying to fix the problem at the algorithms (instead of slapping crude filters on the end of them) would be nice.