4 ms·
There's good reason to be skeptical of AI as it is. Here's a couple of reasons Racial bias in facial recognition: "Error rates up to 34% higher on dark-skinned
by cratermoon 5y ago
There's good reason to be skeptical of AI as it is. Here's a couple of reasons
Racial bias in facial recognition: "Error rates up to 34% higher on dark-skinned women than for lighter-skinned males. "Default camera settings are often not optimized to capture darker skin tones, resulting in lower-quality database images of Black Americans" https://sitn.hms.harvard.edu/flash/2020/racial-discrimination-in-face-recognition-technology/ https://sitn.hms.harvard.edu/flash/2020/racial-discriminatio...
Chicago’s “Heat List” predicts arrests, doesn’t protect people or deter crime: https://mathbabe.org/2016/08/18/chicagos-heat-list-predicts-arrests-doesnt-protect-people-or-deter-crime/ https://mathbabe.org/2016/08/18/chicagos-heat-list-predicts-...
- mgraczyk 5y agoIt's very easy to fix these problems though. There's nothing inherently broken about the models or direction that prevents error rates from being made more uniform. In fact newer facial recognition models with better datasets do perform approximately equally well across skin tones and sex
- cratermoon 5y agoEasy to fix technically, but first the issue must be recognized and demonstrated, then the delicate process of negotiating the social and economic realities in which the technology operates. And that's the problem with ML in general: its failure to recognize the implicit biases in choice of dataset and training and the resulting problems, of which Microsoft racist chatbot Tay[1] is merely the most blatantly ludicrous. 1 https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-revealed-the-dangers-of-online-conversation https://spectrum.ieee.org/in-2016-microsofts-racist-chatbot-...
- mgraczyk 5y agoAnd the first cars didn't have seatbelts. It's fine, these are not complicated problems, and they are much easier to spot and fix than most problems in software engineering at scale. Don't be fooled by the negative PR campaigns and clickbait, there's no reason to be skeptical about ML in general because of this. Also, Tay attempted to solve a much harder problem than image classification. It's hard to build a safe hyperloop. It's no longer hard to build a safe microwave oven.
- SamoyedFurFluff 5y agoForgive me, because I’m not an expert in ML. If this is an easy problem to solve why is it still a problem years after it’s so widespread that msm both knows about it and have written continual investigative journalism about it? It’s clearly not cutting edge anymore once it gets to that point and yet it’s still a problem. Why?
- Mehdi2277 5y agoIt's some work, but not hard to solve technically having been at companies that deal with very similar problems. The main difficulty is less technical and more investment needed vs value + investment is partly outside of the modeling engineers making the system. Part of the improvement can be done by classical computer vision techniques. But mixing classical computer vision techniques with modern ones both feels somewhat like a hack and complicates the system. The other big area though is dataset improvement. Engineers building ml systems and the people collecting and organizing the needed datasets are normally different people with mild connections to each other. For companies that rely mostly on existing datasets and finetune from them, having to add a data curation process is a big pain point. Most companies have immature data curation processes. Many of the popular open source ml datasets have poor racial diversity. The most popular face generation dataset is celebA, full of celebrities (mostly white ones). Other issue is for many of these systems having a racial bias in the error rate has mild business impact which makes it harder to prioritize in fixing. Last issue the work needed to fix this tends to be less interesting than most of the other work to make the system. So overall, the main issues are lack of good open source fair datasets with loose licensing, cross organizational need to solve it (engineers can not code up a fair dataset), and business prioritization. edit: Also solve here is getting accuracy across races to be close not zero. ML models will always have an error rate and if your goal is 0 errors related to racial factors that is extremely hard. Modeling is about making estimates of data not knowing the truth of that data.
- mgraczyk 5y agoOverfitting is also a technically easy problem to solve, but high profile cases in which it's not solved with obvious negative consequences could also lead to investigative journalism.
- pbhjpbhj 5y agoI'm curious how the physics of light is termed racial bias, it's skin-colour bias if anything -- you can be "black" and be lighter skinned than a "white" person, for example -- but surely it's a consequence of how cameras/light works rather than a bias. Of course if you don't take account of the difficulties that come with using the tool then you might be acting with racial bias, but that's different. Or, all cameras/eyes/visual imaging means are "racist".
- cratermoon 5y agoWell, if you really want to know, I have done the research and can recommend several other papers in addition to the one linked. The short answer is that it's not "just physics", and choices made by the chemists and technicians at Kodak, Fuji, Ilford, Agfa, etc to decide how films depicted skin tones were made with racial bias. Digital imaging built on the color rendering tools and tests that originated in the film industry, and thus inherited their flaws.
- pbhjpbhj 5y agoSure, that would be interesting to read about - it would be weird not to adjust your film sensitivity according to market, of that were possible. Always happy to learn, link me up.
- kaba0 5y agoIsn’t that just human bias seeping through into the data set, so of course the neural net trained on that will show similar biases. The problem here is the human element.
- cratermoon 5y agoAs they say, it's not a technical problem, it's a people problem. But it's not "just" human, it's that the field in general is elevating ML, AI, whatever you want to call it, with hype like "algorithms aren't biases like a human would be", which is technically true, but also trivial. The people creating these systems didn't even consider that they would reflect and even enshrine, with all kind of high-priest-of-technology-blessings, their biases, that's why we got Tay and why things PredPol is terrible. The key is to acknowledge and actively protect against systematic bias, not make a business of it (coughtwitterfacebookcough).