14 ms·
Facebook Apologizes After A.I. Puts “Primates” Label on Video of Black Men
- code_duck 5y agohttps://archive.is/dGwHC https://archive.is/dGwHC
- perryizgr8 5y agoAren't we all primates?
- leephillips 5y agoYes, we are, and the author of the NYT article seems weirdly unaware of that. However, the issue here, I’m sure, is that Facebook’s algorithms don’t classify videos of white humans as involving “primates” as well.
- cf100clunk 5y agoThis. The entire point of the article is to highlight what was an unexpected, unanticipated, and most unfortunate AI error. This is a learning experience, pun intended.
- fallingknife 5y agoBut it isn't an error. Humans are primates.
- Veen 5y agoIt is not incorrect by a dictionary definition, but there are other ways of being incorrect. If I identified you at a party by saying "fallingknife is the ape sitting on the sofa" you would probably interpret it differently than if I had said "fallingknife is the person sitting on the sofa". Both are correct, but most people wouldn't interpret them as equally polite or friendly. Members of the set "person" are also members of the superset "ape", but that's irrelevant because the connotations are different, and it's the connotations that are annoying people.
- fallingknife 5y agoThe offense would be entirely based on the fact that I know you know the correct label, and there is emotional processing behind your word choice, neither of which is true in the case of AI.
- xwdv 5y agoWe don’t really know that. It’s possible at some point the AI may have been classifying white humans as primates but nobody cares because it’s white humans. You only have a story worth reading when it’s against an underprivileged minority.
- tgv 5y agoIt's unlikely. Search for images of primate, and you'll see fairly dark, humanoid shapes. Poor training data then seems like a more probable reason for visual confusion. > You only have a story worth reading when it’s against an underprivileged minority. To some extent. But I think there would also be a story if some medicine would work badly on white people, or men. The problem here is that these labels would be there for everyone to see and comment on. If facebook would live by their own rules, they would ban the AI team from using fb accounts.
- amanaplanacanal 5y agoYeah, white primates are pretty weird. It only happened because a few migrated north into temperate Europe and Asia and lost their melanin a while ago.
- xwdv 5y agoThey didn’t lose it. Lighter tones were more popular with potential mates and resulted in gradual whitening over time.
- wizzwizz4 5y agoRunaway sexual selection effects generally don't result in beneficial traits; reduced melanin is beneficial in less sunny climates, because cholecalciferol (so-called “Vitamin D₃”, even though it's not a vitamin) synthesis requires sun exposure. And yes, having less of something is correctly called “losing” it in the English language. That was not a correction.
- wizzwizz4 5y ago
- deleted 5y ago[deleted]
- jonplackett 5y agoThere’s a time and a place for being pedantic, this isn’t it.
- Clampower 5y agoExactly, we should keep feeding the outrage machine.
- krapp 5y agoWhat "outrage machine?" I don't see anyone being outraged. The article doesn't project or describe any outrage, no one in this thread seems to be outraged. (yet.)
- bryan0 5y agoThere is a world of nuanced discussion between flippant dismissal and outrage. Please let’s try to shoot for somewhere in the middle.
- chipotle_coyote 5y agoNo, we should put effort into not being outrageous, whether or not it is intentional.
- jonplackett 5y agoTaking something seriously is what I’m suggesting, not being outraged. Being flippant or intentionally over pedantic is dismissing it as unimportant or a non-issue. It would be easy as a white person (as I am) to look at this and say, ah well it’s just an honest mistake. But this is just an example of a wider problem of AIs being trained on white people. A while back I made the first automatic face swap app Face Juggler and I would often get complaints that it didn’t work as well on black people - I was using apple’s face detection so couldn’t do anything about it which was frustrating. It might seem trivial but it isn’t. AI’s are going to be running more and more stuff that is a lot more important.
- Griffinsauce 5y agoCome on, you know why that is irrelevant here.
- deleted 5y ago[deleted]
- wwweston 5y agoThere's a history of emphasizing a closer relationship of some racial groups to the rest of the animal kingdom as a way of othering them or implying their nature/capabilities are more animal-like and less human-like. So while it's technically true all humans are primates, the selective emphasis and its connotations is an issue. (Weirdly, there's some substantial overlap between people who take offense at the idea they're related to other primates and feel a little freer to emphasize the relationship of a human out-group to other primates, which possibly illuminates something about the problems.)
- saltedonion 5y agoI applaud you to join Facebook PR team. You might be the cure the internet needs where we gain the critical momentum for antitrust action.
- Borrible 5y agoMost primates are not intelligent enough to admit this to themselves. They see themselves as something categorically different. Which, by the way, is related to the discussed problem. The embarrassing thing about these phenomena is that a machine holds up a mirror to us. Naked and ugly. To 'patch' the mirror won't change a thing. Especially when he is right under the right circumstances. In the right context.
- Shorel 5y agoThe angry answers to your comment, and also the fact that the AI has this problem, seems to highlight the USA centric character of the data used and the people working on Facebook. I wonder what would an AI trained with pictures from India would show.
- chuckee 5y ago> The video, dated June 27, 2020, was by The Daily Mail and featured clips of Black men in altercations with white civilians and police officers. It had no connection to monkeys or primates. Huh. All the other incidents the NY Times article mentions come with links, but this one (that the article is ostensibly about) doesn't. Odd.
- ceejayoz 5y ago> The Daily Mail and featured clips of Black men in altercations with white civilians and police officers. It had no connection to monkeys or primates. I'm guessing it ingested the Daily Mail's comment sections.
- daenz 5y agoDon't these kinds of companies have a basic checklist before releasing features like this? And if so, why doesn't this checklist include things like "doesn't identify members of racial group as animals" ? It makes me think that any woke PR coming from Facebook is lip service, if they don't have even basic checks in place.
- saltedonion 5y agoThe space for potential edge cases are so large it will be wack mole. Inference tasks isn’t like software testing where the states are well defined.
- dontreact 5y agoYeah, but it shows some real lack of care to fall into the same edge case that Google was widely criticized for 3 years ago.
- bryan0 5y agoI don’t think this is an edge case. We are asking AI to classify images and videos of people where the results can be disastrous.
- daenz 5y agoA large and complex task is well suited for the talented people behind Facebook. But even a list of basic searches that exercised the engine would have caught this. No excuses imo, particularly since it happened to Google[0] not that long ago. Was nobody paying attention? 0. https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai https://www.theverge.com/2018/1/12/16882408/google-racist-go...
- ta8902 5y agoMaybe they thought it would be racist if they explicitly devised rules preventing black people being identified as primates.
- grawprog 5y agoNot really wanting to delve into the touchy issue of this particular ai fuckup, but it got me thinking about the issue of using ai to compartmentalize and categorize things in general. This is a very obvious example of how bad it can be. Regular complaints from people about ai recommendation engines and most other similar ai driven tagging systems exist. What i wonder though is on a more subtle level, how these ai categorizers are quietly altering our perceptions of things and the world? We notice when it's something obvious like the story in the OP, but how many quiet little mistakes do we just ignore or not notice that on a subconcious level are changing the way we think about or classify things. More and more it seems like society and all things are being neatly placed in little boxes, where they're filed, stamped, indexed, and numbered by fancy algorithms operating without much human oversight, until something like this happens. How much of this is subconciously affecting human perception of the world?
- throwaway81523 5y agoWonder if the AI could be trained with pictures of priests: https://www.ecupatria.org/2020/11/24/the-primate-of-church-of-cyprus-commemorated-the-primate-of-church-of-ukraine-in-the-diptychs/ https://www.ecupatria.org/2020/11/24/the-primate-of-church-o... https://en.wikipedia.org/wiki/Primate_(bishop) https://en.wikipedia.org/wiki/Primate_(bishop)
- dontreact 5y agoSurprised they weren't more careful with this after Google had the same problem a few years ago. https://www.theverge.com/2018/1/12/16882408/google-racist-gorillas-photo-recognition-algorithm-ai https://www.theverge.com/2018/1/12/16882408/google-racist-go...
- bryan0 5y agoYeah you’d think think they be extra careful on their model which classifies videos of people before releasing it.
- dralley 5y agoWhy would you think the "move fast and break things" company would be more careful?
- hn_throwaway_99 5y agoOK, I'll try to tread carefully here, but I think it's really helpful to separate the technical issues from the societal issues. In the Google photo case, I've seen the picture in question, and it's not difficult an all to see how a rudimentary statistical algorithm would mistake the photo for a gorilla. No, no human would make this mistake, but due to the lighting in the photo and the woman's hair it's really not hard to see how that mistake would be made, similar to how Tesla's AI can mistake the sun, low on the horizon, as a yellow traffic light. The algorithm was not "racist", it just didn't have the social context that describing black people as apes and monkeys has a long racist history. And that is really the fundamental problem with most AI these days - it can get very good at statistical inference, but it doesn't have the background knowledge and logic to be able to "think" in the same way humans do. In a similar vein, I recall someone lamenting a couple years ago how Google's street directions would never mispronounce "Malcolm X Boulevard" as "Malcolm 10 Boulevard" if they had any black programmers. Again, given 99% of the time when you see "X" in an address you'd pronounce it as 10, it's not hard to see how this could happen. The problem is that some errors are much more offensive than others, and AI can't really reason about that.
- 5y ago
- A4ET8a8uTh0 5y agoTo me it is just a reminder that AI based picture recognition is not a perfect science and that I should fight any and all attempts to include it in places where it could cause most harm ( policing comes to mind ).
- fallingknife 5y agoBut in this case the label is correct. edit: Hahahaha! Artificial intelligence correctly labels human as member of category "primate." Human "intelligence" flags comment because it doesn't know it is a member of category "primate"
- aaaaaaaaaaab 5y agoI wonder how it would classify this bald chimp: https://www.flickr.com/photos/amandaweeding/3024567966 https://www.flickr.com/photos/amandaweeding/3024567966
- dustymcp 5y agoIt would detect golum, MY PRECIOUS!
- deleted 5y ago[deleted]
- sbmassey 5y agoAccording to the article, the video in question "featured clips of Black men in altercations with white civilians and police officers.", so the primates label could just as well apply to the white civilians or police.
- fallingknife 5y agoSo? The label is correct. I have only contempt for people who are offended by the fact that they are a member of a biological group that includes apes.
- antiquark 5y agoCorrect. https://en.wikipedia.org/wiki/Primate#Classification_of_living_primates https://en.wikipedia.org/wiki/Primate#Classification_of_livi...
- deleted 5y ago[deleted]
- kazinator 5y ago"Primates" is an entirely correct label for any video depicting humans. If you find a picture of my white face anywhere, feel free to put a "primate" caption under it.
- Ekaros 5y ago"Animals" apply too. Also quite a few others.
- deleted 5y ago[deleted]
- amielucha 5y agoAren't people primates?
- deleted 5y ago[deleted]
- WanderPanda 5y agoQuite interesting that this destinction seems to need special care systemically (I believe Google and Twitter had similar issues)
- zaptrem 5y agoThis tends to happen when your dataset isn’t well balanced.
- pvaldes 5y agoAll of us are primates, so what?
- ErikVandeWater 5y agoPeople are saying that this is a very difficult problem to solve. It is not. The problem is not classifying people as "animals;" the problem is classifying people as a known stereotype. Make a list of every animal/thing that correlates to a known stereotype in humans. Then program your AI to never classify those animals/things. It is not a big deal if your AI isn't able to detect gorillas. That doesn't really affect UX. I will add that just because it is an easy problem to solve does not mean they should solve it. An AI classifying someone as something else should be something to laugh at, not take offense to.
- jl6 5y agoThis particular AI failure mode keeps surfacing, but seems to be fixed rapidly when it generates bad press. How do they actually correct it? Teach it some more granular categories of living things? Slap a sensitivity filter on top of its output?
- 6gvONxR4sf7o 5y agoThere’s an amazing number of “but people are primates” comments. Correctness here includes specificity. If person and primate are both candidates, then primate isn’t a correct answer to what this is trying to do. You wouldn’t react to the phrase ‘treating women as objects’ by saying people are objects, right?
- ekabod 5y agopeople are primates is proven whereas women are objects is unproven. So your analogy doesn't work.
- Y_Y 5y agoYou make a good point, and I'd agree that it's implicit that they mean "non-human primates". Otherwise they could solve this by labelling lots of humans as "primates" in the dataset. On the other hand, I think there's a semantic difference between your two cases. I'd argue that treating a woman as an "object" is a different sense of the word to how women (and men and teapots) are literally "objects".
- jonnycat 5y agoThe people here saying the label is "correct" are both wrong, and missing the point. The objective of the model is not "taxonomic classification", the objective is "video recommendation". The label, in the context of the purpose of the model, is completely wrong.
- co2benzoate 5y agoWhy does there tend to be a reflexive set of predictable comments on any article about racism within the tech industry that boils down to "Here is some half baked unresearched explanation that justifies why this instance of racism isn't that big of a deal?"
- UncleMeat 5y agoBecause it permits people to take no action. If people admit there is a problem then they must justify their inaction. But if there is no problem then their inaction is justified be definition.
- Tagbert 5y ago“Aren't people primates?” Words have meaning on multiple levels. On a technical level, yes, all humans are primates, but if the algorithm were following this rule, it would tag all humans in all photos as primates, but it didn’t do that. It only tagged black people. That takes us to a social meaning level. In this context, you have to consider the long, racists history of dehumanizing black people. They have often been called “monkey” and “ape” with the intent to degrade and to segregate them from “people”. In that light, and the fact that the label was not applied in all cases where appropriate, it is hard to see it as anything but racist. Probably not deliberately but if the training material contained racist material, it could bias the result. Even if the training material was inadvertently unbalanced and contained very few black faces, the algorithm would have less experience with black faces and would not know how to categorize them as well as white faces. This is analogous to people who grow up and live largely isolated from close associates with those of other races. Their experiences are filled with varied samples of interactions with people of their own race, but are poor in samples with other races. In the case where we have less experience, we tend to fall back on broad stereotypes and our reactions show strong racial bias.
- Y_Y 5y agoI've dealt with a lot of badly labelled datasets, but I can't say I ever saw one maliciously labelled by racists. There are other more subtle problems that may contribute. As an amateur photographer I've been told that most cameras are calibrated for light skin. If you aren't careful with your setup you'll get bad definition in shadows and relatively very dark areas. If you have badly calibrated/exposed photos then very dark faces, whether of sub-Saharan people or gorillas, may be poorly distinguished. I bet there are loads of unfair, hard-to-catch issues like that. By the way I don't mean to say that you're wrong, but just to offer a Hanlon's Razor-type counterpoint as it occurred to me.
- dagmx 5y agoThis is usually a good example of societal bias compounding. It's possible that nobody directly involved in this was actively racist, but we're building on centuries of racism in every facet. So a little bit of bias in each technology and data set involved, not to mention the engineers and their testing, just adds up
- Borrible 5y agoWhy not just ask the AI what she meant? LIME - Local Interpretable Model-Agnostic Explanations comes to mind. As far as I do understand the classification was the tagging 'primate' on a video of people with different color of skin and different facial features. The interpretation, that our Lady AI meant harm to the people with dark skin color with tagging them as primates is a loading of the viewer. By the way,´a great new opportunity for cyber lawyers and machine inquisitors.
- coretx 5y agoMaybe the AI was right and referring to the police behavior as "primates". We all know how chimps can behave towards other chimps from another population. We humans are not much different.
- Shorel 5y agoThere was also a video of an altercation between black and white people, and the prompt still enquired if it should save the video under "Primates" category. The title has obviously been editorialized for outrage.
- diskzero 5y agoI would like to know if the model was poorly trained or if this was caused by a malicious individual modifying the algorithm to generate offensive suggestions.
- cxmaqq 5y agoOh boy. Who wants to train such a model? Imagine Google used this topic instead of their infamous cross walk recognition captchas.
- tartoran 5y agoI think the technology is very limited but instead it’s overused in cases that shouldn’t. No matter what model is used and how much it’s trained, we have to come to our senses and realize that it will never be perfect and should’t be used everywhere. It’s shudders me to think that this technology is used in legal cases to make life affecting decision that could be absolutely wrong
- axiosgunnar 5y agoWhat if some black people simply look similar to certain types of human-looking apes when photographed at an extreme angle, with a certain lighting, etc? I‘m sure I could take a picture of myself that made me look like a teapot, if I just find the right crop, positioning, etc. Cry racism too many times, and it will lose its strength when some actually racist does happen. Be careful!
- pcurve 5y agoWas the recommendation logic corrupt based on inputs from racist individuals who responded / shared this video on social media? Or is the recommendation logic completely insulated from outside influence? Here's the full video. https://www.facebook.com/watch/?v=2683336318580365 https://www.facebook.com/watch/?v=2683336318580365
- NicoJuicy 5y agoPut primate ( text) next to a white guy and no one would consider it as racist really, because it's correct. But next to a black person it's totally different. Never mind history, i knew it was bad. But I wonder about the effects about "putting it to the center of attention" all the time. Can we fix and name problems that are based on culture and not on race anymore. For example: What would be the consequences of having a statement "i don't know of any problems with Asians who migrated to my country". While i say Asians, it's probably that the culture of them probably has less conflicts? Eg. In our prison ( not US), there is a over representation of certain foreign cultures, according to the numbers. How can we tell if that observation is factual correct and there is a problem with different cultures or if it's a consequence of racism. If that observation would be correct and there is a problem. How can we treat it, since we are not allowed to name it. ( Just to be sure, i am no racist. I'm just wondering out loud and it's very hard to find a place to properly discuss something like this. Which is part of the problem, no?)
- symlinkk 5y agoIs anyone actually upset about this or is this just news sites trying to get clicks? Anyone that thinks about it for half a second would realize this that this is just another case of image recognition being imperfect, just like when reCAPTCHA asks you to select all pictures of buses and there aren’t any. It’s not designed to be racist or malicious so why are we talking about it?
- jrm4 5y agoBlack person here. So, the lack of outrage here (I mean in hacker news) is unsurprising, and would of course be greater if there were more black people "in tech" and by extension here. And even as I write this, I'm not trying to judge -- just more open the door to understanding, like, y'all understand this fact, right? I find myself searching for metaphorical equivalents. I'm thinking something like "what if it labeled Jewish people as rats" or something similar. Even if that doesn't work -- I suppose the broader point I'd try to make here is: A lot of y'all (and me, often) assume the following very wrong idea, subconsciously -- "Because I'm working with a computer I can assume that what we are doing is infused with objectivity, like math" Important to remember nooooooooope. This is dead wrong. We're ALWAYS bringing our subconscious whatevers to this and there is very little, if ANYTHING at all, that can be correctly considered "coldly and obviously correct and object like math is." (perhaps even what we imagine math to be)
- hodgesrm 5y agoGlad you posted. A lot of AI code simply cannot be trusted. In particularly, it's pretty obvious that the teams using it either don't do a lot of QA or don't understand how to test their systems in the first place. I find it mind-boggling that this technology is so widely used with such limited understanding. The interesting question is how the hell do we fix this?
- jpambrun 5y agoOne example doesn't support your claim that QA was not sufficient. You will always get misclassifications, most won't be offensive to anyone. This is not a medical device or a self driving car, this is Facebook trying to squeeze more engagement..
- hodgesrm 5y agoThis type of misclassification is offensive and for good reason. Similar errors have happened regularly for a long time and in many different systems. [1] The comments about enlarging training sets illustrate the blinders that many technical people have about this technology. There does not seem to be a notion of independent quality assurance to detect what should by now be obvious problems. [1] https://www.bostonmagazine.com/news/2018/02/23/artificial-intelligence-race-dark-skin-bias/ https://www.bostonmagazine.com/news/2018/02/23/artificial-in... Edit: oops, left out source
- AHappyCamper 5y agoOh this simply made my day!!!
- overgard 5y agoI'm not an expert, but aren't pretty much all neural networks that are used for classifying images prone to adversarial attacks and general weird conclusions? In this case it comes up with something offensive, but I'm sure there are plenty of other cases where it's just incorrect etc. I'd imagine the only solution would be to have a human editor before these are posted, or just not use this technology in that kind of context.
- nzeribe 5y agoAt this point, it's obvious this is deliberate.