4 ms·

> Research has repeatedly shown that facial recognition technology is fundamentally biased against women and people of color, leading to errors like this. It’s

> Research has repeatedly shown that facial recognition technology is fundamentally biased against women and people of color, leading to errors like this. It’s not fundamentally biased. Many implementations have had imbalanced training sets, which is a different conclusion. But you can’t say CNNs are fundamentally biased against women and people of color. That’s just plain wrong and makes the entire argument look poor.

mandmandam 5y ago
The training sets are biased because of the :fundamental: limitations of current digital imaging technology. Ergo -
- azinman2 5y ago
  No, they were biased because they literally excluded certain races or had small numbers of certain groups. These training sets classically either came from celebrities, or researchers themselves - unbalanced classes of input.
soco 5y ago
Correct, they should remove the word "fundamentally" and then the argument probably holds. Because from what I read those training sets were quite often imbalanced. So I must wonder, where are the training sets coming from? Is it actually the same set, being reused? Is there somewhere a job as "training set creator"?
Pinegulf 5y ago
The source they cite is from Feb 2018 (https://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-0212 https://news.mit.edu/2018/study-finds-gender-skin-type-bias-...) At least review of state of the tech is in order.
- azinman2 5y ago
  And that is very well known at this point and all the big folks are taking measures to counter this.
mjburgess 5y ago
The same datasets given to human operators would not produce biased results. And it isn't merely a question of unbalanced classes. The "bias" is, in part, arising from the inability of machine algorithms to apply rich models to datasets; and to treat "alike images" "alike" -- because the machine has no model of "alike". In the case of images, luminosity values on an image sensor are not constitutive of human facial identity. Any algorithm unable to derive actual identity markers from these is necessarily fundamentally limited -- and routinely, these limitations lead to bias.
- scotty79 5y ago
  > The same datasets given to human operators would not produce biased results. Actually they would. That's where the "all Asians look the same" comes from. Skewed training set.
  - mjburgess 5y ago
    The issue is the limited ability to fit a morphological model to facial features which are "distinct from the ones you are used to". Machines aren't fitting a morphological model. They are being fed 2D pixel patterns -- this information is profoundly insufficient. "Contrast bias in pixel patterns" is not the same as "needing a few more cues to fit a morphological model". The former can never be made into an example of the latter; and therein is the problem.
    - azinman2 5y ago
      It’s as insufficient as a human looking at a photo. We don’t outlaw humans looking at surveillance videos, so why is a machine fundamentally any more biased here?
      - mjburgess 5y ago
        Machines are simply using co-occurance relationships in pixel patterns. Humans have rich models of other humans which we "fit" to the extremely shallow data present in a photograph. This "fitting" is prior to our perception, so as soon as you're perceiving you're also using vast amounts of prior knowledge to reconstruct your environment. The actual "photos" contain almost no information at all -- and lead to gross "stupid" errors.
rsynnott 5y ago
I mean, I'd read that as "facial recognition technology products which actually exist in the real world are fundamentally biased". The platonic ideal facial recognition technology wouldn't be, but it doesn't exist and will never exist so is hardly worth worrying about.
- azinman2 5y ago
  But that’s not even true anymore. It’s well known and counter measures have been taken.
rscoots 5y ago
Everyone mentions 'training sets' in 2021 as if the experts in this field have not been well aware of this issue for decades. Even balanced training sets produce disparate levels of accuracy between races. It would seem computers are just less capable of recognizing certain faces. I've never heard it also affects women though.

mandmandam 5y ago

The training sets are biased because of the :fundamental: limitations of current digital imaging technology. Ergo -

azinman2 5y ago
No, they were biased because they literally excluded certain races or had small numbers of certain groups. These training sets classically either came from celebrities, or researchers themselves - unbalanced classes of input.

soco 5y ago

Correct, they should remove the word "fundamentally" and then the argument probably holds. Because from what I read those training sets were quite often imbalanced. So I must wonder, where are the training sets coming from? Is it actually the same set, being reused? Is there somewhere a job as "training set creator"?

Pinegulf 5y ago

The source they cite is from Feb 2018 (https://news.mit.edu/2018/study-finds-gender-skin-type-bias-artificial-intelligence-systems-0212 https://news.mit.edu/2018/study-finds-gender-skin-type-bias-...) At least review of state of the tech is in order.

azinman2 5y ago
And that is very well known at this point and all the big folks are taking measures to counter this.

mjburgess 5y ago

The same datasets given to human operators would not produce biased results. And it isn't merely a question of unbalanced classes. The "bias" is, in part, arising from the inability of machine algorithms to apply rich models to datasets; and to treat "alike images" "alike" -- because the machine has no model of "alike". In the case of images, luminosity values on an image sensor are not constitutive of human facial identity. Any algorithm unable to derive actual identity markers from these is necessarily fundamentally limited -- and routinely, these limitations lead to bias.

scotty79 5y ago
> The same datasets given to human operators would not produce biased results. Actually they would. That's where the "all Asians look the same" comes from. Skewed training set.
- mjburgess 5y ago
  The issue is the limited ability to fit a morphological model to facial features which are "distinct from the ones you are used to". Machines aren't fitting a morphological model. They are being fed 2D pixel patterns -- this information is profoundly insufficient. "Contrast bias in pixel patterns" is not the same as "needing a few more cues to fit a morphological model". The former can never be made into an example of the latter; and therein is the problem.
  - azinman2 5y ago
    It’s as insufficient as a human looking at a photo. We don’t outlaw humans looking at surveillance videos, so why is a machine fundamentally any more biased here?
    - mjburgess 5y ago
      Machines are simply using co-occurance relationships in pixel patterns. Humans have rich models of other humans which we "fit" to the extremely shallow data present in a photograph. This "fitting" is prior to our perception, so as soon as you're perceiving you're also using vast amounts of prior knowledge to reconstruct your environment. The actual "photos" contain almost no information at all -- and lead to gross "stupid" errors.

rsynnott 5y ago

I mean, I'd read that as "facial recognition technology products which actually exist in the real world are fundamentally biased". The platonic ideal facial recognition technology wouldn't be, but it doesn't exist and will never exist so is hardly worth worrying about.

azinman2 5y ago
But that’s not even true anymore. It’s well known and counter measures have been taken.

rscoots 5y ago

Everyone mentions 'training sets' in 2021 as if the experts in this field have not been well aware of this issue for decades. Even balanced training sets produce disparate levels of accuracy between races. It would seem computers are just less capable of recognizing certain faces. I've never heard it also affects women though.