8 ms·

Darker skin has less contrast between highlights and shadows so isn’t it a more difficult problem to identify them from a technical perspective?

deleted 7y ago
[deleted]
cameronfraser 7y ago
Well the consensus from the article is that middle aged white men had the best rate of success with facial recognition which sounds like a data mismatch problem to me.
shadowgovt 7y ago
It seriously depends on the technology being used. There isn't any evidence from human psychology that dark-skinned people have harder-to-recognize faces, so we're probably just using the wrong approach in the technology currently available.
- formercoder 7y ago
  Great point, I don’t know how much our brain uses, for example, depth information in facial recognition.
- jlawson 7y ago
  Eyes are just better at distinguishing similar dark colors against each other compared to cameras. Only the most expensive, massive and heavy cameras (costing hundreds of thousands of dollars, generally) can compete with eyes in this respect. Plus human facial recognition is massively overpowered for normal cases. I think if you tested it in challenge cases (through blurry video, or snow, or in the dark, or at a distance), you might see darker skin tones being recognized less easily. It'd be an interesting experiment.
  - shadowgovt 7y ago
    That might very well be true, but if it is, then it re-enforces the point: the technology may not be where it needs to be yet, and if it isn't, the tool probably shouldn't be deployed in public use cases where the end-result could be inherently discriminatory by race.
- aaron695 7y ago
  >There isn't any evidence from human psychology that dark-skinned people have harder-to-recognize faces Citation? Dark cars have more accidents- https://www.telegraph.co.uk/motoring/news/7845366/Black-cars-more-likely-to-be-involved-in-crashes.html https://www.telegraph.co.uk/motoring/news/7845366/Black-cars... There clearly will be a difference. Whole careers are around making things more or less recognizable depending on color. What study makes you think it's insignificant for human faces?
  - shadowgovt 7y ago
    What study makes you think it is significant for human faces? I'm noting absence of evidence; if you're making the point that it doesn't imply evidence of absence, I agree, but I've also never heard anyone seriously claim "black people's faces are just harder to recognize." The anecdotal claim isn't even present. The linked study on cars is analogous at best, but doesn't prove anything for human facial recognition. Cars on highways and human beings in various settings are extremely different circumstances.
- SiVal 7y ago
  Part of the technology is the capturing of photons, fewer of which bounce off dark skin than light skin, so it is in part the same generic problem as always: fewer photons, less data, worse images. Whether it's day vs. night, fast lens vs. slow, close vs. distant, fast moving vs slow (requiring fast shutter than lets in less light), or whatever causes it, fewer photons mean more problems. And even if you get enough photons to, for example, recognize both black and white equally well in daylight, you might not at night or in a deep shadow indoors, or up close equally well but there will be some distance at which the advantage of more photons begins to matter, or some speed, or some combination of factors. Any technological improvements will help, but there won't be any technological solution that will always work equally well with fewer photons as with more.
- ummonk 7y ago
  Cameras don't have the same dynamic range as human eyes.
  - ailideex 7y ago
    Maybe there is some transitive effect from racially biased cameras.
  - notauser 7y ago
    The very best HDR cameras have roughly the same dynamic ranges as the human eye. You can't display all of this range to people because of the limitations of display technology, but you can feed the full range to a facial recognition engine. Security budgets might not stretch to top-end HDR equipment but the price keeps on coming down. The performance of a modern flagship phone is remarkable compared to a few years ago - and fixed surveillance cameras can have much bigger glass and sensors, making it cheaper to get super-human performance. One new but related issue is that body-worn cameras can capture more low-light detail than the human eye. Police unions have argued against deploying these sensors, because, they want the evidential record to show what the officer could see - not what a cat could see.
    - randomcarbloke 7y ago
      Most digital cameras have a dynamic range well in excess of the human eye.
- etaioinshrdlu 7y ago
  Less reflected light, all else being equal, means the same amount of detected noise (whether using cameras or eyes), and therefore a worse signal to noise ratio on the dark image. Worse signal to noise ratio means less information. Less information indicates worse detection accuracy. While there isn't evidence that dark-skinned people have harder-to-recognize faces, there is also no evidence to the contrary. It seems like the null hypothesis in this case, given no additional evidence, is to assume that darker images are indeed harder to recognize.
kevingadd 7y ago
To a degree this comes down to how you capture an image of their skin. Lighting has a big impact on how the result looks as does the medium you use to capture the image. In the early days of color photography there was an issue with some films and reference images being tuned for the most common subject (light-skinned humans) and as a result if you tried to capture a mix of races you'd get bad results: https://petapixel.com/2015/09/19/heres-a-look-at-how-color-film-was-originally-biased-toward-white-people/ https://petapixel.com/2015/09/19/heres-a-look-at-how-color-f... This makes sense if you consider how light is (generalizing here) a broad spectrum of hues and a given material is most reflective for specific parts of the spectrum, so if you don't capture much there you'll get a low-contrast image, like stripping the R channel out of an RGB bitmap. It's of course possible to solve the problem for a wider set of skin tones, and it has been solved, but it takes more work. It's a subject of ongoing discussion/experimentation in film to this day: https://www.konbini.com/en/cinema/insecure-cinematographer-how-light-black-skin/ https://www.konbini.com/en/cinema/insecure-cinematographer-h...
rolltiide 7y ago
the point is that these products would not have shipped at all if their workforces were diverse enough to point out “hey this doesn't work, like at all” or “hey this glaring edge case isn't far enough on the edge to send this to production”
- okusername 7y ago
  That's a bold assumption. Such algorithms are developed wwith training sets, not just by employees randomly clicking around. They wrote that 189 algorithms were submitted, there were surely a few developed by non-racist, diverse people.
  - rolltiide 7y ago
    the hyperbole is not really helpful or substantive, its not about the algorithm its about how the product wouldnt have shipped when individual contributors, their manager, the director, the executive team, the board, and their vendors all noticed that it didnt work very well when the demo failed because they were not white males testing it on themselves and their colleagues and photos of their friends
  - omar_a1 7y ago
    It's the methods being used to select, populate, label, and validate the training set that are the problem. Basically, your team is composed of white dudes who don't see the problem with a ML training set consisting largely of pictures of white dudes. To prevent this they'd have needed to A) Employ a black person, and B) Listen to said employee's feedback, in order to recognize the problem. Edit: Also worth pointing out, just using a representative population sampling would still show racial bias, essentially weighting accuracy with respect to population percent. You'd probably need to have equal samplings of pictures of people from all races/genders/disabilities if you wanted equal accuracy across the board. That also includes picture quality and range of picture quality. Doubling up images, or using corporate headshot white dudes and grainy selfie People of Color could still cause issues. Same logic applies to labelling. That minimum wage contracting firm used to decide who's who in the photos may exhibit racial bias, by virtue of the fact that most people do. If their accuracy in labelling is racially biased then so too will the algorithms that it's based on. In short: Racist garbage in, racist garbage out.
deleted 7y ago
[deleted]
trianglem 7y ago
Doesn’t this depend on the backdrop.
- formercoder 7y ago
  Complicated answer. Theoretically no, I was just mentioning highlights vs shadows on the face which are properties of reflectivity. The backdrop certainly matters from a camera exposure perspective in order to optimize for the skin properties in play.

deleted 7y ago

[deleted]

cameronfraser 7y ago

Well the consensus from the article is that middle aged white men had the best rate of success with facial recognition which sounds like a data mismatch problem to me.

shadowgovt 7y ago

It seriously depends on the technology being used. There isn't any evidence from human psychology that dark-skinned people have harder-to-recognize faces, so we're probably just using the wrong approach in the technology currently available.

formercoder 7y ago
Great point, I don’t know how much our brain uses, for example, depth information in facial recognition.
jlawson 7y ago
Eyes are just better at distinguishing similar dark colors against each other compared to cameras. Only the most expensive, massive and heavy cameras (costing hundreds of thousands of dollars, generally) can compete with eyes in this respect. Plus human facial recognition is massively overpowered for normal cases. I think if you tested it in challenge cases (through blurry video, or snow, or in the dark, or at a distance), you might see darker skin tones being recognized less easily. It'd be an interesting experiment.
- shadowgovt 7y ago
  That might very well be true, but if it is, then it re-enforces the point: the technology may not be where it needs to be yet, and if it isn't, the tool probably shouldn't be deployed in public use cases where the end-result could be inherently discriminatory by race.
aaron695 7y ago
>There isn't any evidence from human psychology that dark-skinned people have harder-to-recognize faces Citation? Dark cars have more accidents- https://www.telegraph.co.uk/motoring/news/7845366/Black-cars-more-likely-to-be-involved-in-crashes.html https://www.telegraph.co.uk/motoring/news/7845366/Black-cars... There clearly will be a difference. Whole careers are around making things more or less recognizable depending on color. What study makes you think it's insignificant for human faces?
- shadowgovt 7y ago
  What study makes you think it is significant for human faces? I'm noting absence of evidence; if you're making the point that it doesn't imply evidence of absence, I agree, but I've also never heard anyone seriously claim "black people's faces are just harder to recognize." The anecdotal claim isn't even present. The linked study on cars is analogous at best, but doesn't prove anything for human facial recognition. Cars on highways and human beings in various settings are extremely different circumstances.
SiVal 7y ago
Part of the technology is the capturing of photons, fewer of which bounce off dark skin than light skin, so it is in part the same generic problem as always: fewer photons, less data, worse images. Whether it's day vs. night, fast lens vs. slow, close vs. distant, fast moving vs slow (requiring fast shutter than lets in less light), or whatever causes it, fewer photons mean more problems. And even if you get enough photons to, for example, recognize both black and white equally well in daylight, you might not at night or in a deep shadow indoors, or up close equally well but there will be some distance at which the advantage of more photons begins to matter, or some speed, or some combination of factors. Any technological improvements will help, but there won't be any technological solution that will always work equally well with fewer photons as with more.
ummonk 7y ago
Cameras don't have the same dynamic range as human eyes.
- ailideex 7y ago
  Maybe there is some transitive effect from racially biased cameras.
- notauser 7y ago
  The very best HDR cameras have roughly the same dynamic ranges as the human eye. You can't display all of this range to people because of the limitations of display technology, but you can feed the full range to a facial recognition engine. Security budgets might not stretch to top-end HDR equipment but the price keeps on coming down. The performance of a modern flagship phone is remarkable compared to a few years ago - and fixed surveillance cameras can have much bigger glass and sensors, making it cheaper to get super-human performance. One new but related issue is that body-worn cameras can capture more low-light detail than the human eye. Police unions have argued against deploying these sensors, because, they want the evidential record to show what the officer could see - not what a cat could see.
  - randomcarbloke 7y ago
    Most digital cameras have a dynamic range well in excess of the human eye.
etaioinshrdlu 7y ago
Less reflected light, all else being equal, means the same amount of detected noise (whether using cameras or eyes), and therefore a worse signal to noise ratio on the dark image. Worse signal to noise ratio means less information. Less information indicates worse detection accuracy. While there isn't evidence that dark-skinned people have harder-to-recognize faces, there is also no evidence to the contrary. It seems like the null hypothesis in this case, given no additional evidence, is to assume that darker images are indeed harder to recognize.

kevingadd 7y ago

To a degree this comes down to how you capture an image of their skin. Lighting has a big impact on how the result looks as does the medium you use to capture the image. In the early days of color photography there was an issue with some films and reference images being tuned for the most common subject (light-skinned humans) and as a result if you tried to capture a mix of races you'd get bad results: https://petapixel.com/2015/09/19/heres-a-look-at-how-color-film-was-originally-biased-toward-white-people/ https://petapixel.com/2015/09/19/heres-a-look-at-how-color-f... This makes sense if you consider how light is (generalizing here) a broad spectrum of hues and a given material is most reflective for specific parts of the spectrum, so if you don't capture much there you'll get a low-contrast image, like stripping the R channel out of an RGB bitmap. It's of course possible to solve the problem for a wider set of skin tones, and it has been solved, but it takes more work. It's a subject of ongoing discussion/experimentation in film to this day: https://www.konbini.com/en/cinema/insecure-cinematographer-how-light-black-skin/ https://www.konbini.com/en/cinema/insecure-cinematographer-h...

rolltiide 7y ago

the point is that these products would not have shipped at all if their workforces were diverse enough to point out “hey this doesn't work, like at all” or “hey this glaring edge case isn't far enough on the edge to send this to production”

okusername 7y ago
That's a bold assumption. Such algorithms are developed wwith training sets, not just by employees randomly clicking around. They wrote that 189 algorithms were submitted, there were surely a few developed by non-racist, diverse people.
- rolltiide 7y ago
  the hyperbole is not really helpful or substantive, its not about the algorithm its about how the product wouldnt have shipped when individual contributors, their manager, the director, the executive team, the board, and their vendors all noticed that it didnt work very well when the demo failed because they were not white males testing it on themselves and their colleagues and photos of their friends
- omar_a1 7y ago
  It's the methods being used to select, populate, label, and validate the training set that are the problem. Basically, your team is composed of white dudes who don't see the problem with a ML training set consisting largely of pictures of white dudes. To prevent this they'd have needed to A) Employ a black person, and B) Listen to said employee's feedback, in order to recognize the problem. Edit: Also worth pointing out, just using a representative population sampling would still show racial bias, essentially weighting accuracy with respect to population percent. You'd probably need to have equal samplings of pictures of people from all races/genders/disabilities if you wanted equal accuracy across the board. That also includes picture quality and range of picture quality. Doubling up images, or using corporate headshot white dudes and grainy selfie People of Color could still cause issues. Same logic applies to labelling. That minimum wage contracting firm used to decide who's who in the photos may exhibit racial bias, by virtue of the fact that most people do. If their accuracy in labelling is racially biased then so too will the algorithms that it's based on. In short: Racist garbage in, racist garbage out.

deleted 7y ago

[deleted]

trianglem 7y ago

Doesn’t this depend on the backdrop.

formercoder 7y ago
Complicated answer. Theoretically no, I was just mentioning highlights vs shadows on the face which are properties of reflectivity. The backdrop certainly matters from a camera exposure perspective in order to optimize for the skin properties in play.