10 ms·
Semantics derived automatically from language corpora contain human-like biases
- randomwalker 10y agoCoauthor here. Some of the press articles about our work didn't have a lot of nuance (unsurprisingly), but in the paper we're careful about what we say, what we don't say, and what the implications are. Happy to engage in informed discussion :)
- fpp 10y agovery interesting topic - could you please share a link to the original paper. best probably to read that one first.
- privong 10y ago> very interesting topic - could you please share a link to the original paper. Unless the link was changed in the few minutes since you posted your comment, the link for the article is the original Science paper (http://science.sciencemag.org/content/356/6334/183.full http://science.sciencemag.org/content/356/6334/183.full)
- pasbesoin 10y ago(From a Javascript-disabled perspective) Page with actual link: http://science.sciencemag.org/content/356/6334/183/tab-pdf http://science.sciencemag.org/content/356/6334/183/tab-pdf Link to PDF itself: http://science.sciencemag.org/content/sci/356/6334/183.full.pdf http://science.sciencemag.org/content/sci/356/6334/183.full....
- fpp 10y agosorry mixed it up. If I look at glove & WordNet usage e.g. for topic extraction, bagging / clustering or semantic similarity would you say we would need to get rid of such a bias, e.g. create something like a Geiger counter for NLP. Alternative view - when doing sentiment analysis / classification would you say that such a bias actually helps to identify a type of sentiment in a doc / sentence.
- russdpale 10y agoWouldn't this lead to an entire of idea of contextual bias? Times when it could benefit and be used, and times where it is occluded.
- glibgil 10y agoCan you provide an example of how this bias might play out in a human-AI interaction?
- andreasvc 10y agoIt appears that the linked paper has examples.
- yummyfajitas 10y agoThe paper has them: - An AI correctly infers (simply by reading text) that a physicist is male and a nurse is female. - An AI correctly infers the gender of humans with androgyonous names. - An AI infers insects are unpleasant and flowers are pleasant to humans. - An AI also infers that African American names are more likely to be associated with unpleasantness than European names. [edit: to those who dislike this comment, can you tell me what you object to? Which of my concrete examples is not in the paper?]
- yummyfajitas 10y agoDo you have any evidence that this effect results in machines making systematically wrong inferences? Near as I can tell, your paper shows that these "biases" result in significantly more accurate predictions. For example, Fig 1 shows that a machine trained on human language can accurately predict the % female of many professions. Fig 2 shows the machine can accurately predict the gender of humans. Normally I'd expect a "bias" to result in wrong predictions - but in this case (due to an unusual redefinition of "bias") the exact opposite seems to occur. (Drawing on your analogy with stereotypes, it's probably also worth linking to a pointer on stereotype accuracy: http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-unbearable-accuracy-of-stereotypes.pdf http://emilkirkegaard.dk/en/wp-content/uploads/Jussim-et-al-... http://spsp.org/blog/stereotype-accuracy-response http://spsp.org/blog/stereotype-accuracy-response )
- andreasvc 10y agoI think your questions would be answered by reading the article. Particularly: "In AI and machine learning, bias refers generally to prior information, a necessary prerequisite for intelligent action (4). Yet bias can be problematic where such information is derived from aspects of human culture known to lead to harmful behavior. Here, we will call such biases “stereotyped” and actions taken on their basis “prejudiced.”" This definition is not unusual. This is about inferences that are wrong in the sense of prejudiced, not necessarily inaccurate.
- yummyfajitas 10y agoThe usual definition of bias in ML papers is E[theta_estimator - theta]. That is explicitly a systematically wrong prediction. In any case, the paper suggests that this "bias" or "prejudice" is better described as "truths I don't like". I'm asking if the author knows of any cases where they are actually not truthful. The paper does not suggest any, but maybe there are some?
- deleted 10y ago[deleted]
- 10y ago
- EGreg 10y agoIsn't the word bias being redefined by a social justice point of view? Normally bias would be with reference to failing to match reality (eg women in general have physically weaker upper body than men), and not failing to match whatever standard of equality a society wishes were the case eventually.
- sp332 10y agoYou should check out the Implicit Association Test that they used to measure the biases. Just as one example, there's nothing about being a doctor that is inherently more male or female. So all gender differences would have external causes.
- ylem 10y agoHi! I just read the paper--impressive work! Have you tried any other languages? For example, French or German?
- slackstation 10y agoSince we think of biases of a large human corpora as wrong, I'm curious how one would find one or make one that is "right". Given how accurate human corpora is at predicting things like gender distribution in jobs for instance, wouldn't making an "unbiased" corpora make an inaccurate AI? Shouldn't we be careful in implying things like the biases and solutions to said biases? For instance, I'd like to know if my algorithm for filtering job applicants is trying to undo the injustices of the world in addition to finding the best candidates.
- reader5000 10y agoIt's very amusing SJWs getting triggered by AIs.
- deleted 10y ago[deleted]
- draw_down 10y agoShut up.
- dang 10y agoWhoa, posting like this is a bannable offence on Hacker News. Your recent comment history looks pretty civil, so we won't ban you, but really please don't do this.
- rjbwork 10y agoCan we somehow see comments that are removed? If not, are there any plans to be transparent about which speech is removed from HN? I would rather a lock subthread type functionality rather than full on deletion. On reddit I find myself drawn to the bottom of comments sections often, in order to see the dissent and/or anger and the prevailing responses, whereas here I've noticed more lately of removals and bans all the way at the bottom.
- necessity 10y agoMixing sjw is in the abstract, must be shit. Didn't read.
- throwaway91111 10y agoBe careful with the word truth. You don't know what it means.
- yummyfajitas 10y agoSee Fig 1 and 2.
- TeMPOraL 10y agoTruth is a simple concept. http://yudkowsky.net/rational/the-simple-truth/ http://yudkowsky.net/rational/the-simple-truth/
- dang 10y agoWe detached this subthread from https://news.ycombinator.com/item?id=14116195 https://news.ycombinator.com/item?id=14116195 and marked it off-topic.
- mnarayan01 10y ago> That is, before providing an explicit or institutional explanation for why individuals make prejudiced decisions, one must show that it was not a simple outcome of unthinking reproduction of statistical regularities absorbed with language. This is an extraordinarily bold claim. I'd be quite interested in how peoples' responses to the article changed if this was the lead.
- Osmium 10y ago> This is an extraordinarily bold claim. Very bold. The full quote is worth reproducing: > Our results also suggest a null hypothesis for explaining origins of prejudicial behavior in humans, namely, the implicit transmission of ingroup/outgroup identity information through language. That is, before providing an explicit or institutional explanation for why individuals make prejudiced decisions, one must show that it was not a simple outcome of unthinking reproduction of statistical regularities absorbed with language. I'm reminded of Parable of the Polygons[0] which illustrates Shelling's model of segregation, showing how quite small initial biases can be amplified and result in very large segregation. It would be seem very sad if tribalism in all its forms is simply an emergent behaviour, a result of a random fluctuation (e.g. one or two racist individuals) causing a chain reaction throughout society, where biases become gradually amplified even if most individuals are, themselves, generally well-meaning. How do we escape from that? [0] http://ncase.me/polygons/ http://ncase.me/polygons/
- TeMPOraL 10y ago> It would be seem very sad if tribalism in all its forms is simply an emergent behaviour, a result of a random fluctuation (e.g. one or two racist individuals) causing a chain reaction throughout society, where biases become gradually amplified even if most individuals are, themselves, generally well-meaning. How do we escape from that? Sadly, that seems to be the case. I wouldn't even turn racism into a "special case" here. Humans are capable of dividing themselves into ingroups and outgroups over everything, no matter how trivial. I suspect the segregation process will occur with all in/outgroup divisions. Separations along the race and gender lines are particularly prevalent because those are the most obvious, noticeable differentiators between people.
- Houshalter 10y agoThe paper and title implies it's absorbing these stereotypes from humans. I think there is another explanation. Remember these models are trained on a dataset of news or Wikipedia articles. And it's 'goal' is to find vectors that predict what contexts words are more likely to appear in. So if 34% of doctors are female, then you would expect 34% of doctors in news or Wikipedia articles to be female. Even if the articles are completely unbiased and the writers have no stereotypes whatsoever. And so the word vector would naturally label "doctor" something like "66% likely to occur in a male context". And in fact this paper confirms that. Figure 1 shows that the word vectors are highly predictive of the actual gender distribution of various occupations. Probably much more accurate than most people would be. So it's not mindlessly absorbing human stereotypes. It's learning reality's stereotypes. This result is completely expected and desirable. What makes word vectors so powerful is how they can learn complicated correlations between words and their contexts. The famous example is how it learns that "Queen" is the female equivalent of "King". Which is a gender stereotype as well. If it wasn't able to learn that doctors were a bit more likely to be male, that would be more surprising.
- andreasvc 10y agoI think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (indirectly) absorbing stereotypes from humans. This is an expected result, but not necessarily desirable in the end.
- yummyfajitas 10y agoThe parent's point is that they may not be absorbing stereotypes from humans at all. They may be generating accurate beliefs about the world from text representations of the world.
- rmxt 10y agoSo, "plants are pleasant" or "insects are unpleasant" are universal truths?
- Avshalom 10y agohttps://www.youtube.com/watch?v=cfRDUsvu5fE https://www.youtube.com/watch?v=cfRDUsvu5fE
- yummyfajitas 10y agoPositively right is a statement about beliefs. Normatively right is a statement about actions. It's my belief that if you want an ML system to take normatively right actions, you should explicitly encode the value of those actions (or the world states they are designed to achieve, if you are more utilitarian than virtue ethics) into it's utility function. To give a concrete example from the area of lending, you shouldn't build a bot that can't accurately infer that blacks don't pay back their loans as much as whites. You should instead assign an explicit dollar value to how much you want to lend to blacks, and maximize profit + that dollar value. I.e., "lending to a black person is worth $500 to me". My takeaway from this paper is that bots accurately learn accurate stereotypes from language, and word embeddings do a great job of learning true facts about the world. But some people wish they didn't, because there are things man should not know.
- dang 10y ago> because there are things man should not know I get that you like to take contrarian positions. But you also make a habit of inserting flamebait into your posts about them. This combination is trolling. If you continue to do this we will ban you. Specifically, we need you to stop playing the following game on HN: 1. Post contrarian view 2. Include provocation 3. People get provoked 4. Act like people can't handle your truth Accidental trolling—e.g. triggering a flamewar with an unintended turn of phrase—is venial. But when you consistently generate such effects, you become responsible, regardless of what's wrong with others or their views. You passed that line on HN a long time ago. I'm told that self-responsibility is a conservative value and even recall you posting many criticisms of people whom you consider not to practice it. Please practice it here. We detached this comment from https://news.ycombinator.com/item?id=14116469 https://news.ycombinator.com/item?id=14116469 and marked it off-topic.
- yummyfajitas 10y agoYou're wildly mischaracterizing what I said. I didn't suggest "people can't handle my truth". I suggested the article we are discussing says there are certain truths that ML systems should not learn. The article explicitly recommends building systems which can't learn those things, and suggests characterizing them is a first step: "We recommend addressing this through the explicit characterization of acceptable behavior. One such approach is seen in the nascent field of fairness in machine learning, which specifies and enforces mathematical formulations of nondiscrimination in decision-making (19, 20). Another approach can be found in modular AI architectures, such as cognitive systems, in which implicit learning of statistical regularities can be compartmentalized and augmented with explicit instruction of rules of appropriate conduct (21, 22)." According to the article, the most closely related work "is concurrent work by Bolukbasi et al. (6), who propose a method to “debias” word embeddings." (Recall that in the context of the article, "bias" is something that generates true predictions that are objectionable rather than something which is false.)
- ramblenode 10y agoThis was a neat study, but the authors really do throw around a number of speculative and unsupported claims in the discussion about linguistic relativity/determinism. > Our findings are also sure to contribute to the debate concerning the Sapir-Whorf hypothesis (17), because our work suggests that behavior can be driven by cultural history embedded in a term’s historic use. Such histories can evidently vary between languages. No, not really. Linguistic relativity makes a claim about the direction of causation--from language to thought. This study does nothing to test the direction of causation, which is usually considered possible only with controlled experiments. This chicken and egg debate has been going on for the better part of the last century--it is not an easy problem. > Our results also suggest a null hypothesis for explaining origins of prejudicial behavior in humans, namely, the implicit transmission of ingroup/outgroup identity information through language. That is, before providing an explicit or institutional explanation for why individuals make prejudiced decisions, one must show that it was not a simple outcome of unthinking reproduction of statistical regularities absorbed with language. Once again, this was not an experiment capable of producing causal evidence. The authors have shown that human biases can be replicated by statistical learning of language corpora--admirable work, but nothing new here for Sapir-Whorf.