3 ms·
To assume that dark skinned people aren't involved in creating or testing facial recognition, or that women aren't, seems bizarre and vaguely racist/sexist. It
by onion2k 7y ago
To assume that dark skinned people aren't involved in creating or testing facial recognition, or that women aren't, seems bizarre and vaguely racist/sexist.
It's unlikely anyone making a facial recognition system would choose to release a version that didn't work, at the very least, for everyone on the team.
If the team is only white men then you can see why they might accidently release something that only works on white men's faces. If there was a woman or a PoC on the team they'd have someone saying "Hang on, it doesn't work on me!"
The important thing to learn here is that any system is only ever as good as the data used to test it. If your test data sucks then the system you build is also going to suck.
- jevgeni 7y agoAlso, aren't facial recognition algos trained on broadly available data sets and NOT on the bunch of developers who are working on it? A sample of on average five people seems to be a dumb way to train facial recognition - whether diverse or not.
- onion2k 7y agoAlso, aren't facial recognition algos trained on broadly available data sets... They should be, yes, but those training sets can have biases too, and if you use them your app will have the same biases. There are moves to make things like diverse data sets for this exact problem - https://www.theverge.com/2018/6/27/17509400/facial-recognition-bias-ibm-data-training https://www.theverge.com/2018/6/27/17509400/facial-recogniti... A sample of on average five people seems to be a dumb way to train facial recognition - whether diverse or not. I was talking about testing rather than training. You can train your facial recognition model on a diverse range of faces, but if you only test it on a few faces you don't know if it works for diverse people. Teams of developers who test using their own data (eg faces) and nothing else is annoyingly common.
- ALittleLight 7y agoThis might be a valid theory if facial recognition was a single niche product made by unsophisticated or under resourced developers. That's incredibly far from reality though. There are many well funded groups of serious professionals and academics working on facial recognition with data sets of millions of faces. It's just objectively not correct and not reasonable to think that the reason facial recognition works better on some demographics than others is because nobody thought of training models on those demographics. If you believe that is the case, then do you think people working on facial recognition reading this blog post are slapping themselves on the forehead saying "Of course. Train the models on someone besides the dev team!"?
- onion2k 7y agoI was talking about testing the application rather than training the model. You can train a ML model on a diverse range of faces, but if, for example, an application that uses the model doesn't calibrate the camera properly it still won't work for darker faces.
- ALittleLight 7y agoI don't understand. The different accuracy of facial recognition by demographic group is common across multiple facial recognition systems. It's highly unlikely that all of the affected systems owe their inaccuracy to a post-training configuration issue. To me it seems more like you are arguing for the sake of arguing rather than that you actually believe in your point. You are just trying to conceive of imaginative possibilities that preserve the "dumb facial recognition developers forgot non-white people exist" interpretation. The problem is that your imagined possible alternatives are not plausible explanations.
- onion2k 7y agoI gave a single example of why an app might fail at facial recognition in some cases. Obviously I wasn't suggesting that would be the case for every app that doesn't work. Suggesting I was is a strawman of ridiculous proportions. I also didn't say anything about ML training data in the post you replied to. My point was about facial recognition app developers not testing with a diverse range of inputs, which is why we see apps failing in public when people try to use them with inputs the app hasn't been tested against. If the apps were well tested those cases would have been caught before the public ever saw them. My original point stands - if your testing sucks then it's very likely that your app will suck too. There will be problems with it that you don't know about, and the users will find them very quickly.
- thu2111 7y agoDo you have any data showing facial recognition doesn't work well on black people? My understanding is that this is an urban legend - it's not true, the algorithms work fine assuming a decent quality camera and video (same for everyone).
- travisoneill1 7y agoThe incentives of engineers building facial recognition have nothing to do with whether or not the model can recognize the particular faces of the team members. I don't believe this argument that the reason that facial recognition is racially biased because of the race of the team members. The reason that I don't believe this is because it is written about so often that there will have been many engineers who have built models with training data sets equalized by race to eliminate this bias, and yet the difference persists.