6 ms·
I think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (in
by andreasvc 9y ago
I think you are making a distinction without a difference. If the word vectors pick up biases from wikipedia text, than for all practical purposes, they are (indirectly) absorbing stereotypes from humans. This is an expected result, but not necessarily desirable in the end.
- yummyfajitas 9y agoThe parent's point is that they may not be absorbing stereotypes from humans at all. They may be generating accurate beliefs about the world from text representations of the world.
- rmxt 9y agoSo, "plants are pleasant" or "insects are unpleasant" are universal truths?
- Houshalter 9y agoQuite possibly. Words relating to insects will occur in news articles about malaria, zika, crop destruction, etc. Words relating to plants might occur in articles about arbor day, spring time, environmentalism, etc.
- pharrington 9y agoAn exercise: Words relating to insects will occur in news articles about environmentalism, crop production, rituals of rebirth, etc. Words relating to plants might occur in articles about crop destruction, the international drug trade, people getting poisoned, etc. rmxt questioned the universality of sentiment analysis. Responding by noting specific contexts, free from a clear coherent general structure, is an assertion against the discovered sentiments' universal truth.
- Houshalter 9y agoBut it is a universal truth that humans generally find plants pleasant and insects unpleasant. And the word "pleasant" is entirely based on human preferences after all.
- pharrington 9y agoWhat I'm probably missing indeed is that scoping of universality to humans. Lately I've been trying to be more explicit in my written communications in an attempt to understand both the limits of my knowledge and perceptions and the limits of the sources of information that I digest. Is suggesting that pleasantness is a sentiment that's not unique to humans really that controversial? super late edit: it's specifically flowers, not plants, that people are biased towards finding pleasant
- mcguire 9y ago"...it is a universal truth that humans generally find plants pleasant..." Ah, but those exceptions are really unpleasant.
- yummyfajitas 9y agoThe example the parent poster described is "doctors are 66% male", which could very well be true.
- rmxt 9y agoMy point is that neither "insects are unpleasant" nor "plants are pleasant" nor "doctors are 66% male" are immutable features of the universe. They are merely snapshots of the human view of world conditions, as the world is now. "True now", but not "true forever and always". The paper seems to advocate for designing ML systems that learn that what is "true now" may not be "true forever and always". It seems to be quite the opposite of "there are certain truths that ML systems should not learn."
- chongli 9y agoIf your standard for truth is "immutable feature of the universe" then you might as well give up now because we don't know about any of those, or indeed if any exist at all. Setting such a standard for a machine is ridiculous if all you want is a new tool to get some work done.
- deleted 9y ago[deleted]
- pharrington 9y agoThe machines are generating accurate semantic graphs of human generated text representations of human observations.
- scribu 9y agoThe GP is saying that the bias isn't an attribute of the Wikipedia text, but of reality. If the reality is that only 34% of doctors are female, why is it not desirable for the machine to learn that?
- andreasvc 9y agoAgain, from humans and from reality is not different. Whether it is desirable to avoid stereotypes depends on your values.
- scribu 9y agoWhat humans think they know is not actually reality - fine. But then what's the difference between a fact and a stereotype, in your opinion?
- andreasvc 9y agoA fact is a statement corresponding with reality, a stereotype is a belief/generalization deemed harmful/undesirable.
- pharrington 9y agoNot answering for the parent: a fact is any instantaneous snapshot of reality. A stereotype is misapplying properties of specific and limited context to a universal scope.
- jrkatz 9y agoIt depends on what you want the machine to do. If you are making a gambling machine that looks at pairs of names and makes bets as to which name belongs to a doctor, you want it to learn that. If the machine looks at names and decides who to award a "become a doctor" scholarship to, based on who it thinks is most likely to succeed, you don't want it to learn that.
- scribu 9y agoI agree that if your goal is to build a machine that decides who gets to become a doctor, you need to do more than just let it loose on a bunch of text. But I don't think preventing it from learning the current state of the world is a good strategy. Adding a separate "morality system" seems like a more robust solution.
- Houshalter 9y agoThe distinction is very important. If it's just regurgitating human biases that would be bad. Humans often have very inaccurate and warped beliefs after all. If it's accurately modelling reality, then what's the problem? That's what we want it to do. Why would you want a less accurate model of reality? I've seen interpretations of this result that think it's proof "language is sexist" or whatever. But there's no evidence that the humans who wrote the corups had any bias at all. As long as there are more news articles about female nurses than male nurses, the model will learn a correlation between the concepts.
- anigbrowl 9y agoWhy would we want to reproduce existing structures of oppression in mechanical form? Have you noticed how automation often vastly amplifies things? It's a short step from saying 'this model accurately reflects the bias in society' to 'that's how things are, the computer says women aren't cut out to be doctors.' Surely you are aware that in real world world people rationalize decisions they don't actually understand all the time because they are not capable of or interested in improving upon the system within which they pursue their own economic interest on behalf of others whose interests do not seem coincident with their own.
- noir_lord 9y ago> Why would we want to reproduce existing structures of oppression in mechanical form? If (for example) 66% of Doctors are male and 34% female then it's not reproducing "existing structures of oppression" it's inferring something about reality.
- arrrg 9y agoIf it’s saying that and only that then that’s obviously fine. However, if that knowledge is then applied in any other way, then that’s problematic.
- notahacker 9y agoIn an environment in which Blue people are banned from becoming doctors, its also inferring something about reality to conclude that 0% of Doctors are Blue. It would be entirely wrong, however, to use these inputs to infer anything whatsoever about the respective propensity of Blue and Green people to become doctors in an environment in which such a rule or idea of a rule had never existed. Obviously "structures of oppression" - real and imagined - which lead to fewer female doctors even in western liberal democracies where women wishing to become doctors are generally met with encouragement are less extreme, but that isn't to say they don't exist or that a computer output (or human interpretation of said computer output) is likely to draw correct inferences from it. And if you think that people won't use the idea that the outputs are unbiased because the computer isn't programmed with the same prejudices that produce the inputs, I have some algorithmically-generated investment advice involving a bridge to sell you