4 ms·

I understand how ml can replicate existing cultural bias in recommendation systems or risk scoring systems, but how does bias work in the context of facial reco

by kgin 6y ago

I understand how ml can replicate existing cultural bias in recommendation systems or risk scoring systems, but how does bias work in the context of facial recognition?

pmorici 6y ago
Train it with a set of data that doesn't include a wide enough variety of faces.
- kgin 6y ago
  For facial recognition, aren’t you just hoping for multiple images of the same person? A wide variety of faces doesn’t help you match specific people. It’s not about tagging a face as “human” it’s about tagging Bob’s face as “Bob”
  - ineedasername 6y ago
    Yeah, I think the comments here are talking about both using the same terminology without clarifying, which was a bit confusing until I realized that was happening.
  - greycol 6y ago
    But now explain to me the difference between Bob's face and another humans face without referencing what another humans face looks like, or the difference between bob's face in front of a white wall vs a blue sky. Basically to train an ML system you'll need pictures of bob and pictures of not bob. If the pictures of not Bob don't include any humans then there's a good chance that every human face gets classified as Bob because they look a lot more like Bob than an airplane. If the pictures of human faces are only white people and Bob is black then it's quite possible all black faces you show the system after training get classified as Bob.
    - kgin 6y ago
      So the problem is mostly when deciding whether a face is likely one of the faces already in the system or if it is a new face that hasn’t been added yet?
      - greycol 6y ago
        So the problem is it's hard to give a definitive answer because there are many different types of facial recognition systems. ML systems have a phase when you are training the algorithm. When you are adding data in this phase you are altering the algorithm rather than adding data to compare against. For facial recognition it might be learning things like distance between eyes is useful for distinguishing faces, that the wall behind a person is not good for distinguishing people, etc. If this stage of the process goes badly then you may end up with it assuming common characteristics are important than they are. When you add faces after training you are usually not altering the algorithm (some ml techniques do continue learning but there are challenges in this approach too), you are just adding faces the algorithm is generating outputs for. So for a naive face lock algorithm on a phone you may have your trained algorithm that compares two pictures of faces and puts out a number based on how certain the faces are the same person. So all that's happening when the user is 'training' face lock is the adding of a bunch of pictures of their face, then when you go to unlock the unlock algorithm just compares the image from the camera to all of the stored images and if any match closely enough it unlocks. Now let's consider the naive law enforcement approach with a similar algorithm. We'll go with the hypothetical: You load in a bunch of outstanding warrants and whenever your body camera captures an image that matches the image in a warrant it will ding and you can go to the cruiser and pull up the warrant for further scrutiny. If the facial recognition software say placed a large importance on whether a person had an epicanthic fold (because it was enough to be a distinguishing feature in the training data set) then if one warantee was of asian descent and had a big nose then all it might take for others of the same descent to have a high match with that person is to have a big nose, even if their other features are distinctly different. Adding more pictures of the wanted man wouldn't actually help this it would infact make it more likely that more asians would be held up while the warrant was checked by a human. Now you can be smarter about your logic around the output from your ML algorithm but to actually get better output to base your logic on you actually need to train the ML better in the first place (or retrain it). That requires data, good data for what your trying to classify, actually figuring out what that data should contain and making sure it contains it is a huge task in itself. A publicly available corpus may not be as robust as required for this training and there is little incentive for private companies to share robust data as this can be considered the secret sauce in their offerings as well as being a barrier to entry to competitors.
  - pmorici 6y ago
    I had this story about HP webcam software in mind. [0] But I'd say it very much depends on what you are doing. I don't think it is an inherent short coming of the technology but if you don't think about what you are doing the result might give the wrong perception. [0] https://www.cnn.com/2009/TECH/12/22/hp.webcams/index.html https://www.cnn.com/2009/TECH/12/22/hp.webcams/index.html
  - baddox 6y ago
    I don’t think that’s necessarily true. I don’t know much about machine learning, but I’m pretty sure a common technique is to use a model that is trained on a huge number of relevant data, and that model outputs a fingerprint of a given input, and the fingerprints of two inputs can be used to estimate the similarity of those two inputs.
- jerf 6y ago
  If this was the root problem, it isn't that hard a problem to solve, and a lot of people have the motivation to solve it and gather the justified social plaudits for solving the problem. Is it seriously the case that nobody has collected a more diverse face set despite it being public knowledge for years that this is a problem? It doesn't take that long.
Barrin92 6y ago
technically much lower accuracy on African-American and Asian populations (in the US)[1]. More importantly technical issues aside the primary use case of facial recognition seems to be the policing of minority or vulnerable populatoins and erosion of privacy more broadly, which tends to hit minorities the worst. [1]https://www.wsj.com/articles/facial-recognition-software-suffers-from-racial-bias-u-s-study-finds-11576807304 https://www.wsj.com/articles/facial-recognition-software-suf...
- deleted 6y ago
  [deleted]
- partiallypro 6y ago
  But isn't the bias mostly just because of the lack of data? Asian facial recognition works very well (or very badly, depending on your perspective) in China...where there is a ton of data on Asian facial structures, etc, for example.
  - Traster 6y ago
    It's not just a problem of lack of data, it's a problem of composition of data. If there's a signal that increases your accuracy for asian faces and decreases it for caucasian faces, in systems deployed in China the system weights will be adjusted one way, in systems deployed in Europe you'll get the other way, and Asians in Europe and Caucasians in China will get bad performance. Maybe people don't care about that, but wait until that performance difference is between black and white people in America.
rockinghigh 6y ago
I see at least two main sources of bias: 1) your training data does not have enough people with dark skin or African American face features, 2) your hardware and image processing pipeline relies on color contrast or skin brightness. One way to measure bias is to compare the average error rare across ethnic groups.
- centimeter 6y ago
  > your training data does not have enough people with dark skin or African American face features This isn’t the issue - the issue is lower variance across black faces in any basis. > your hardware and image processing pipeline relies on color contrast That is how vision works.
  - brnt 6y ago
    You keep repeating black faces have lower variance, but you never back this up. Personally I find it a preposterous thesis, I find the opposite to be true. I know for a fact genetic diversity is highest among Africans. By being so convinced of this without showing any proof you are running to risk of proving the point at the root of this issue: we are not addressing our bias in building the system. Not only in our data, but in your case perhaps even not in our thinking.
    - centimeter 6y ago
      > I know for a fact genetic diversity is highest among Africans. This doesn't mean what you think it means. In particular, it does not imply wider phenotypic variance.
      - brnt 6y ago
        No, but it makes it more likely. Meanwhile, you had a nights sleep and still show us nothing.
  - cycomanic 6y ago
    You continue to bring this up without evidence. Sounds to me you're just stating your own prejudice see https://en.m.wikipedia.org/wiki/Cross-race_effect https://en.m.wikipedia.org/wiki/Cross-race_effect
chillacy 6y ago
Well, humans have a version of this bias as well: https://en.wikipedia.org/wiki/Cross-race_effect https://en.wikipedia.org/wiki/Cross-race_effect > Thus, to the uninitiated Caucasian, all East Asian people look alike, while to East Asian people, all Caucasian people look alike
- gnulinux 6y ago
  I definitely had this issue when I was growing up in a Balkan country. I couldn't differentiate Asian faces, I remember having a lot of difficulty with this while watching movies. Fast-forward a decade, now I'm living in the US, and I can differentiate Asian faces as easy as White faces or Black faces. For me, it was all a matter of "training" (and by that I mean interacting with people with these facial features).

by kgin 6y ago

I understand how ml can replicate existing cultural bias in recommendation systems or risk scoring systems, but how does bias work in the context of facial recognition?

pmorici 6y ago

Train it with a set of data that doesn't include a wide enough variety of faces.

kgin 6y ago
For facial recognition, aren’t you just hoping for multiple images of the same person? A wide variety of faces doesn’t help you match specific people. It’s not about tagging a face as “human” it’s about tagging Bob’s face as “Bob”
- ineedasername 6y ago
  Yeah, I think the comments here are talking about both using the same terminology without clarifying, which was a bit confusing until I realized that was happening.
- greycol 6y ago
  But now explain to me the difference between Bob's face and another humans face without referencing what another humans face looks like, or the difference between bob's face in front of a white wall vs a blue sky. Basically to train an ML system you'll need pictures of bob and pictures of not bob. If the pictures of not Bob don't include any humans then there's a good chance that every human face gets classified as Bob because they look a lot more like Bob than an airplane. If the pictures of human faces are only white people and Bob is black then it's quite possible all black faces you show the system after training get classified as Bob.
  - kgin 6y ago
    So the problem is mostly when deciding whether a face is likely one of the faces already in the system or if it is a new face that hasn’t been added yet?
    - greycol 6y ago
      So the problem is it's hard to give a definitive answer because there are many different types of facial recognition systems. ML systems have a phase when you are training the algorithm. When you are adding data in this phase you are altering the algorithm rather than adding data to compare against. For facial recognition it might be learning things like distance between eyes is useful for distinguishing faces, that the wall behind a person is not good for distinguishing people, etc. If this stage of the process goes badly then you may end up with it assuming common characteristics are important than they are. When you add faces after training you are usually not altering the algorithm (some ml techniques do continue learning but there are challenges in this approach too), you are just adding faces the algorithm is generating outputs for. So for a naive face lock algorithm on a phone you may have your trained algorithm that compares two pictures of faces and puts out a number based on how certain the faces are the same person. So all that's happening when the user is 'training' face lock is the adding of a bunch of pictures of their face, then when you go to unlock the unlock algorithm just compares the image from the camera to all of the stored images and if any match closely enough it unlocks. Now let's consider the naive law enforcement approach with a similar algorithm. We'll go with the hypothetical: You load in a bunch of outstanding warrants and whenever your body camera captures an image that matches the image in a warrant it will ding and you can go to the cruiser and pull up the warrant for further scrutiny. If the facial recognition software say placed a large importance on whether a person had an epicanthic fold (because it was enough to be a distinguishing feature in the training data set) then if one warantee was of asian descent and had a big nose then all it might take for others of the same descent to have a high match with that person is to have a big nose, even if their other features are distinctly different. Adding more pictures of the wanted man wouldn't actually help this it would infact make it more likely that more asians would be held up while the warrant was checked by a human. Now you can be smarter about your logic around the output from your ML algorithm but to actually get better output to base your logic on you actually need to train the ML better in the first place (or retrain it). That requires data, good data for what your trying to classify, actually figuring out what that data should contain and making sure it contains it is a huge task in itself. A publicly available corpus may not be as robust as required for this training and there is little incentive for private companies to share robust data as this can be considered the secret sauce in their offerings as well as being a barrier to entry to competitors.
- pmorici 6y ago
  I had this story about HP webcam software in mind. [0] But I'd say it very much depends on what you are doing. I don't think it is an inherent short coming of the technology but if you don't think about what you are doing the result might give the wrong perception. [0] https://www.cnn.com/2009/TECH/12/22/hp.webcams/index.html https://www.cnn.com/2009/TECH/12/22/hp.webcams/index.html
- baddox 6y ago
  I don’t think that’s necessarily true. I don’t know much about machine learning, but I’m pretty sure a common technique is to use a model that is trained on a huge number of relevant data, and that model outputs a fingerprint of a given input, and the fingerprints of two inputs can be used to estimate the similarity of those two inputs.
jerf 6y ago
If this was the root problem, it isn't that hard a problem to solve, and a lot of people have the motivation to solve it and gather the justified social plaudits for solving the problem. Is it seriously the case that nobody has collected a more diverse face set despite it being public knowledge for years that this is a problem? It doesn't take that long.

Barrin92 6y ago

technically much lower accuracy on African-American and Asian populations (in the US)[1]. More importantly technical issues aside the primary use case of facial recognition seems to be the policing of minority or vulnerable populatoins and erosion of privacy more broadly, which tends to hit minorities the worst. [1]https://www.wsj.com/articles/facial-recognition-software-suffers-from-racial-bias-u-s-study-finds-11576807304 https://www.wsj.com/articles/facial-recognition-software-suf...

deleted 6y ago
[deleted]
partiallypro 6y ago
But isn't the bias mostly just because of the lack of data? Asian facial recognition works very well (or very badly, depending on your perspective) in China...where there is a ton of data on Asian facial structures, etc, for example.
- Traster 6y ago
  It's not just a problem of lack of data, it's a problem of composition of data. If there's a signal that increases your accuracy for asian faces and decreases it for caucasian faces, in systems deployed in China the system weights will be adjusted one way, in systems deployed in Europe you'll get the other way, and Asians in Europe and Caucasians in China will get bad performance. Maybe people don't care about that, but wait until that performance difference is between black and white people in America.

rockinghigh 6y ago

I see at least two main sources of bias: 1) your training data does not have enough people with dark skin or African American face features, 2) your hardware and image processing pipeline relies on color contrast or skin brightness. One way to measure bias is to compare the average error rare across ethnic groups.

centimeter 6y ago
> your training data does not have enough people with dark skin or African American face features This isn’t the issue - the issue is lower variance across black faces in any basis. > your hardware and image processing pipeline relies on color contrast That is how vision works.
- brnt 6y ago
  You keep repeating black faces have lower variance, but you never back this up. Personally I find it a preposterous thesis, I find the opposite to be true. I know for a fact genetic diversity is highest among Africans. By being so convinced of this without showing any proof you are running to risk of proving the point at the root of this issue: we are not addressing our bias in building the system. Not only in our data, but in your case perhaps even not in our thinking.
  - centimeter 6y ago
    > I know for a fact genetic diversity is highest among Africans. This doesn't mean what you think it means. In particular, it does not imply wider phenotypic variance.
    - brnt 6y ago
      No, but it makes it more likely. Meanwhile, you had a nights sleep and still show us nothing.
- cycomanic 6y ago
  You continue to bring this up without evidence. Sounds to me you're just stating your own prejudice see https://en.m.wikipedia.org/wiki/Cross-race_effect https://en.m.wikipedia.org/wiki/Cross-race_effect

chillacy 6y ago

Well, humans have a version of this bias as well: https://en.wikipedia.org/wiki/Cross-race_effect https://en.wikipedia.org/wiki/Cross-race_effect > Thus, to the uninitiated Caucasian, all East Asian people look alike, while to East Asian people, all Caucasian people look alike

gnulinux 6y ago
I definitely had this issue when I was growing up in a Balkan country. I couldn't differentiate Asian faces, I remember having a lot of difficulty with this while watching movies. Fast-forward a decade, now I'm living in the US, and I can differentiate Asian faces as easy as White faces or Black faces. For me, it was all a matter of "training" (and by that I mean interacting with people with these facial features).