30 ms·
Facebook apology as AI labels black men 'primates'
- q-rews 5y agoI feel that 0-failure-rate expectations from technology will keep us from progressing as a species. Facebook disabled Thai-to-English translation back in April because it translated the queen as “slut” and it’s been disabled since. Maybe we should learn to accept non-fatal errors from applications instead of forcing things to stop entirely. I find it ridiculous that my Photos app suggests I change monkey to “lemur” while I have plenty of photos of monkeys and zero of lemurs.
- yanlezeiler 5y agoI worked for another computer vision company, Clarifai that had the same issue. One of the employees noticed it and we retrained the model before it became public.
- MBCook 5y agoThis is what amazes me. Given this exact thing has happened in the past and resulted in public humiliation of the companies involved, how did they not notice this? Why didn’t they check for it?
- deleted 5y ago[deleted]
- avalys 5y agoWhat’s being reported is that there is a single video which is mislabeled. For all we know, they did test for this, and believed there was no issue. AI models are deterministic in a purely technical sense, but practically speaking, they are non-deterministic black boxes. It’s not as if you can write a unit test which generates all possible videos of black people and makes sure it never outputs “gorilla”.
- MattGaiser 5y agoChecking would require checking every possible video against the classifier.
- sammy2244 5y agoI don’t blame them, black people really do act like primates sometimes. Take south Chicago for example
- supperburg 5y agoSome people have weird faces. Maybe these black guys had strange, guerilla-like faces? Jut some more fodder for the angry mob.
- MBCook 5y agoWow.
- ALittleLight 5y agoThe video features white and black men. It seems like concluding the algorithm is calling black men primates is the same kind of error people are accusing the algorithm/Facebook of. i.e. The reason you think it's racist is because you assume it's talking about black people specifically suggesting you think the word is more apt to describe black people. Primates and humans are similar labels. This was almost certainly not intentional. Video classifiers are going to make mistakes - sometimes crude or offensive ones. I don't get outrage over labeling errors like this. Facebook should fix the issue - but they shouldn't apologize. It only encourages grievance seekers.
- TOMDM 5y agoPrevious discussion from a couple days ago. https://news.ycombinator.com/item?id=28415582 https://news.ycombinator.com/item?id=28415582
- lindenstark 5y agomy mistake, missed that.
- TOMDM 5y agoEh, no harm, 200+ comments and no one complaining. People obviously still see value in discussing it
- mycodesucks 5y agoI mean... It's not wrong, we're all primates.
- tartoran 5y agoHow would you feel if you’d be labeled supremacist (only if your skin complexion was white)? After all the human species is the supreme species... Oh well I guess that wouldn’t bother you and you’d take it as a compliment. Now imagine the reverse of a compliment, to be given one of the labels that has been used again and again to belittle you, to make you feel inferior (only to make others feel above you), a label which you’d get stamped on regardless of any of your acomplishments, qualities, virtues and so on.
- deleted 5y ago[deleted]
- q-rews 5y agoI think the core difference is that one is a honest mistake and the latter is just deprecating. It’s fine if AI calls me “white treehouse” but it’s less fine if it labels me “stupid”, even if my face really looks stupid. The reality is that people have become extremely sensitive about some topics even if they’re not the recipients of the perceived insult.
- tartoran 5y agoI don’t think any algo should be in the business of labeling any people period. What you call sensitive is offensive to others as much as you being labeled stupid by the look of your face. It is wrong in your case and it is also wrong in the cases you perceive as sensitive. I actually find it quite insensitive of you to dictate what others should feel based on your experience. And selfish too
- q-rews 5y ago> I don’t think any algo should be in the business of labeling any people period. You’re proving my point. What you’re saying is that we should stop any algo from categorizing anything just to avoid a 0.00001% of hurting someone’s feelings. People like you are going to bring back the dark ages. Go back to Twitter. Can’t we just treat these incidents as bugs and fix them instead of just fueling the outrage machine?
- scotty79 5y agoIs that a result of a skewed training set or are people really hard to tell apart from gorillas if there are no obvious tells like large difference in brightness of different areas of the face?
- MattGaiser 5y agoI would be interested to know if young children would do much better in many cases. When I was quite young, I referred to some firefighters as robots.
- JasonFruit 5y agoIt reminds me of some of the explorers' tales of people who were half-human, half some other animal, or of people covered in hair, the first of which may have originated from seeing people riding animals, and the others to various (actual) primates. If humans can make such mistakes, certainly Facebook's AI can be excused for its confusion.
- MattGaiser 5y agoApparently we are not sure whether the original term Gorillai referred to people or gorillas. https://en.wikipedia.org/wiki/Gorilla#Etymology https://en.wikipedia.org/wiki/Gorilla#Etymology
- klyrs 5y agoAre you sure those were honest mistakes and not stories for the sake of storytelling? And no, prehistoric ignorance does not justify this system making it to production.
- deleted 5y ago[deleted]
- Barrin92 5y agothe biggest tell is probably that, unless the person is in a zoo, I haven't seen many gorillas at starbucks which says a lot about the state of our alleged human outperforming AI
- smoldesu 5y agoWho takes the fall when an AI screws up? If you shine enough light on it, apparently the brand does. If a human were to do this, the company would immediately fire the employee and cut all ties with them. But as the article points out, 'fixing' an AI mistake isn't really a fix at all: > [Google] said it was "appalled and genuinely sorry", though its fix, Wired reported in 2018, was simply to censor photo searches and tags for the word "gorilla".
- exporectomy 5y agoIf a human did it with intent to offend, they would. But if a black person genuinely looked like a primate to human eyes, that would be pretty shitty to fire the poor worker who had no way of knowing. Here, the AI isn't trying to offend so maybe there should be no consequences and people should stop demanding severe punishment for minor accidental insults.
- fidesomnes 5y agoDo a google image search for "white families" the result might surprise you.
- simonw 5y agoThis happened to both Google Photos and Flickr too. Which makes it an inexcusable mistake to make in 2021 - how are you not testing for this? Google Photos in 2015: https://www.wired.com/story/when-it-comes-to-gorillas-google-photos-remains-blind/ https://www.wired.com/story/when-it-comes-to-gorillas-google... Flickr in 2015: https://www.independent.co.uk/life-style/gadgets-and-tech/news/flickr-s-auto-tagging-feature-goes-awry-accidentally-tags-black-people-apes-10264144.html https://www.independent.co.uk/life-style/gadgets-and-tech/ne...
- bpodgursky 5y agoYou can't just "test" a neural network like that. For all you know they tested a thousand pictures of Chimpanzees and Gorillas against the network, but for some reason the NN decided to classify the photo differently because the subject was standing in front of the wrong kind of tree or wearing a funny-colored hat. There's no super reliable way to prevent this (with current tech) other than forbidding that output entirely.
- IncRnd 5y agoThe reason these companies don't fix these systems is because they don't know how. It is easier to remove certain outputs or retire the whole system. There is no line of code they can tweak.
- nolaspring 5y agoRichen the dataset it’s trained on enough so that the model is correct before you release it to prod.
- throwthere 5y agoMaybe the algo or the training set or something else was racist, maybe it wasn't. But if you code something that labels people slurs, you've messed something up. Like, you need to be 99.999999% sure you're not throwing out slurs or your whole project is failing spectacularly. And then you have to apologize to the 0.0000001% , which is still probably like 10 people if half the planet uses your site. How do you get there? I don't know. I guess it'd help if you could be 99.999999% sure you weren't looking at a human face before using another label. Like, bias towards humans in a big big way. Heck, the pre-test probability that your algo is looking at a person is probably much higher than the one from your training set if you're facebook. Or maybe you drop primates from your training set. I guess in that case you'll misidentify some primates as people-- which is kind of the flipside of the same problem technically but oh so much more acceptable.
- overgard 5y agoThis isn't the kind of slur that you can just run a dictionary search for. There are totally valid contexts to tag a gorilla in a picture if it contains gorillas. I'm sure there are other words it could also mistakenly classify with that might be insulting on accident but arent slurs (maybe tagging an athlete as a statue, for instance, like "that quarterback is a statue in the pocket"). This tech isn't perfect so you either need a human editor or you have to learn to live with mistakes. IMO the fact that this was unintentional and an AI mistake makes me think the outrage is more performative than genuine.
- throwthere 5y agoOh boy. People who know basic ML think "Oh, it was unintentional, just a basic misclassification, it happens." Guess what? You're still calling people slurs on your website, even if you did it accidentally.
- overgard 5y agoI'm not saying it's not bad, but I think as adults we should all be able to differentiate between an accidental insult and an intentional one.
- userbinator 5y agoThe AI is very honest and innocent, it doesn't know what political correctness is. I've heard stories of parents whose kids would also mislabel a black human as a gorilla.
- tartoran 5y agoAI doesn’t build itself in a vaccuum. The very people who train it are likely to be biased in one direction or another and if left unchecked their mastetpiece will be biased as well. Now, if these algos were staying in a lab or something it wouldn’t be a problem but as soon as they hit the real world they should be held up to some standards, don’t you think?
- exporectomy 5y agoPeople need to learn to cope with the difference between innocent mistakes and expressions of genuine feelings of contempt/etc.
- userbinator 5y agoUnfortunately, this is the sort of article that just causes all the SJWs to come crawling out and destroy any attempts at logical reasoning.
- crooked-v 5y agoAn 'innocent mistake' performed by a megacorporation with bajillions of dollars, of exactly the same kind of error that has publicly appeared in the actions of other megacorporations with bajillions of dollars... at that point it's not a 'mistake', it's just the people in charge not caring.
- jimjimjim 5y agonothing important in the world should RELY on a AI/nn/ml.
- SilverRed 5y agoThis feature was not of any importance. It just asks the viewer if they want to see more or less of a certain kind of video.
- jimjimjim 5y agoyes, and now imagine if the feature was important. like sorting job applications, or medical diagnosis, or (dramatic pause) driving. Lots of organizations are looking at ways of completely removing humans from their decision making processes. 99% certainty rates would be fantastic unless you are in the 1% false positive/negative group
- kodah 5y agoI don't really think the world needs AI right now. One can argue that the AI is making an innocent mistake and that calling an AI or ML (or it's improper training, however that works) "racist" is overblown rhetoric as people are here, but I think all of that aschews the actual issue. The problem is that AI and ML are primarily used for decision making, like in recommendation engines. These little gadgets that provide recommendations may be fairly low-stakes, but are theoretically proof-of-concepts for future applications like policing, fighting terrorism, or human trafficking. If you get it wrong there, the consequences are devastating. If people don't raise the flag about how wildly wrong the AI is now, then there will inevitably be a false confidence to use it for the aforementioned applications (and there are plenty of examples of how this has already happened).
- kongin 5y agoAs they should. All humans are primates, just because you're white doesn't mean you're a lizard.
- istillwritecode 5y agoGoogle: hold my beer
- Cycl0ps 5y agoI don't like these stories. It always trends towards the most inflammatory arguments, those being inherint bias and unconscious racism put upon our technology. Real issues in those topics aside, are any articles like this doing anything but feeding flames and generating ad revenue? Instead, I want to talk about pareidolia. Humans are social creatures. We have evolved to identify others of our kind and read their expressions. This was important to us, as we evolved alongside gorilla analogues as well, and the few of us that couldn't discern one face from another didn't usually last long. I think we're trying to place too much of a human expectation onto these machines. I think that human features and primate features are strikingly similar, and it's our specialized brains that let us so easily discern. Yes, with enough data and training we could have more accurate models, but we can't cry foul everytime an algorithm doesn't behave like a human does. Reference: https://www.reddit.com/r/Pareidolia/ https://www.reddit.com/r/Pareidolia/
- OneEyedRobot 5y ago>I don't like these stories. It always trends towards the most inflammatory arguments, those being inherint bias and unconscious racism put upon our technology. Oh well, it's the times we live in. If people simply laughed at the results and fixed the problems they'd miss all the endorphin rush of outrage.
- dimitrios1 5y agoHaven't you heard? Words are literal violence and making me feel unsafe. This cannot stand, and the situation must be rectified, otherwise you are complicit.
- TaylorAlexander 5y agoYou’re engaging in precisely the inflammatory rhetoric you seem to disagree with. EDIT: Hard to tell these days but other comments suggest the parent was being sarcastic.
- deleted 5y ago
- OneEyedRobot 5y agoI wonder if AIs are good at distinguishing individual gorillas, etc. I'd never really thought about the problem of classification being harder (perhaps) than identification if you see what I mean.
- sonicggg 5y agoWow, what else? Did it also label them as "Homo Sapiens"?
- Paraesthetic 5y agoooof, thats uncomfortable
- varelse 5y agoAI is not the problem here. AI just notices stuff. It's the lack of even amateur hour emotional intelligence in the product managers who deploy systems like this IMO.
- firefoxd 5y agoThis happens because there are no black people of consequence in the ML pipeline. In my previous company Everytime we built a new model, a bunch of us would test it. Being the only black person in the company, I often found some very odd things and we would correct it before shipping. I understand that fb is a much bigger scale, but all the reason to have a much more diverse set of eyes to test their models before they go live. If you want to avoid this, hire more black people, seriously.
- SamReidHughes 5y agoThey already get hired, when they meet the hiring bar.
- dshpala 5y ago"Hire more black people" - isn't that what FAANG desperately been trying to do for some time now? I guess first step might be to "hire more black QA people".
- joebob42 5y agoIt's not obvious to me what black people would have done to fix this specific problem. Would they have said "oh we should make sure to test the algorithm on blurry images of people in a forest and make sure it doesn't get confused"?
- firefoxd 5y ago"I uploaded my picture and it says I'm a monkey" "Oh, maybe we should look into that"
- deleted 5y ago[deleted]
- npteljes 5y agoWhat I don't get is that people seriously think that FB/Google don't test with images of black people? Especially after one of such mislabeling scandal happened already? Very highly doubtful.
- root_axis 5y agoI think the negative reaction is reasonable. Clearly, if a human did this it would a problem, so why should it be acceptable for an automated system to do the same thing? The fact that it is unintentional doesn't negate the fact that it's an embarrassing mistake. On the other hand, imagine a world where these labels were applied by a massive team of humans instead of a deep learning algorithm. At Facebook's scale, would the photos end up with more or less racist labels on average over time? My guess is that the model does a better job, but this is just another example of why we should be wary about trusting ML systems with important work.
- ipaddr 5y agoThat raises the question, is it embarrassing or an expected mistake to be learned from. Many things are mislabel many things are labelled properly but we never say AI must feel pride at the good labeling job why would we give emotions to an emotionless system?
- deleted 5y ago[deleted]
- root_axis 5y ago> it embarrassing or an expected mistake to be learned from. I would say it's both. It's embarrassing for Facebook because it looks racist even though it really isn't. The system might be emotionless but the people who interact with it aren't, and we don't expect them to be.
- crooked-v 5y ago> it looks racist even though it really isn't It absolutely is racist. Racist outcomes are still racist regardless of whether there's a guy in Klansmen robes at the steering wheel or not.
- saltyfamiliar 5y agoHow is it a racist outcome? This has nothing to do with the belief that one "race" is inherently better than another. It's a simple categorization error due to insufficient training data.
- silisili 5y agoI've been trying to avoid controversy lately, but hey, here's one to downvote. Have we considered AI and ML as a general brain replacement is a failed idea? That we humans feel we are so smart we can recreate or exceed millions of year evolution of a human brain? I'd never call AI a waste, it's not. But getting it to do human things just may be. Even a child can tell the difference between a human of any color and an ape. How many billions have been spent trying, and failing, to exceed the bar of the thoughts of a human child?
- SilverRed 5y agoNo it isn't a failed idea at all. The products out today are remarkably useful even if not perfect. I have tested out the google lens thing on google photos and it is astounding. I took a photo of the water pump from a car windscreen wiper and google was able to correctly identify what it was. I took a photo of a generic PCB which showed the back of a driver board for an LCD and google was able to bring up the exact type of board it was with the names of the ICs on it. In these examples, google photos ai has far exceeded what the average human can achieve. We just have to keep in mind that these systems are not perfect and only a best guess which should be verified by a person later. The problem here is not that the mistake was very costly or disruptive to the function of the feature, but that the mistake was highly offensive which is something very hard to avoid.
- silisili 5y agoThe problems it solved for you are immensely useful to you, but not remarkable IMO. The problem it's solving is that it can do things that somebody with zero experience cannot. If you had an auto parts pro, or an EE, they probably could have done the same for you. So, in general, AI is helpful because it has a much larger breadth of knowledge. Granted. But I want examples of it doing depth, too. My wife uses Lens when we fish. It's way, way worse than a fisherman with any experience at all.
- SilverRed 5y agoRegardless, it is still far beyond a "failed idea" since it provides genuine value and achieves at a higher level than the average human for many topics. It isn't as you say better than an expert but the fact that for free I get something that works well is remarkable to me considering this is cutting edge technology.
- dd444fgdfg 5y agoHumans are primates. The AI is correct. Does it classify white men and Asians as primates too? If not, that's a bug.