4 ms·
The GP is saying that the bias isn't an attribute of the Wikipedia text, but of reality. If the reality is that only 34% of doctors are female, why is it not d
by scribu 9y ago
The GP is saying that the bias isn't an attribute of the Wikipedia text, but of reality.
If the reality is that only 34% of doctors are female, why is it not desirable for the machine to learn that?
- andreasvc 9y agoAgain, from humans and from reality is not different. Whether it is desirable to avoid stereotypes depends on your values.
- scribu 9y agoWhat humans think they know is not actually reality - fine. But then what's the difference between a fact and a stereotype, in your opinion?
- andreasvc 9y agoA fact is a statement corresponding with reality, a stereotype is a belief/generalization deemed harmful/undesirable.
- pharrington 9y agoNot answering for the parent: a fact is any instantaneous snapshot of reality. A stereotype is misapplying properties of specific and limited context to a universal scope.
- jrkatz 9y agoIt depends on what you want the machine to do. If you are making a gambling machine that looks at pairs of names and makes bets as to which name belongs to a doctor, you want it to learn that. If the machine looks at names and decides who to award a "become a doctor" scholarship to, based on who it thinks is most likely to succeed, you don't want it to learn that.
- scribu 9y agoI agree that if your goal is to build a machine that decides who gets to become a doctor, you need to do more than just let it loose on a bunch of text. But I don't think preventing it from learning the current state of the world is a good strategy. Adding a separate "morality system" seems like a more robust solution.
- rspeer 9y agoWhat do you think of Bolukbasi's approach that's mentioned in the article? In short, you let a system learn the "current state of the world" (as reflected by your corpus), then put it through an algebraic transformation that subtracts known biases. Do you consider that algebraic transformation enough of a "morality system"? I hope you're not saying we shouldn't work on this problem until we have AGI that has an actual representation of "morality", because that would be a setback of decades at least.
- scribu 9y ago> put it through an algebraic transformation that subtracts known biases > Do you consider that algebraic transformation enough of a "morality system"? I would consider it a sort of morality, yes. But keep in mind that the list of "known biases" would itself be biased toward a particular goal, be it political correctness or something else.
- rspeer 9y agoYes, every step of machine learning has potential bias, we know that, that's what this whole discussion is about. Nobody would responsibly claim that they have solved bias. But they should be able to do something about it without their progress being denied by facile moral relativism. If we can't agree that one can improve a system that automatically thinks "terrorist" when it sees the word "Arab" by making it not do that, we don't have much to talk about.
- deleted 9y ago
- notahacker 9y agoEven if a categorization is true in a trivial sense, what generally isn't reported and thus readily inferred from fairly naive text-parsing algorithms is significant. People generally don't bother stating a perpetrator (or indeed a victim or possible witness law enforcement hopes to contact) is $majorityrace in most countries' crime reports, for example.
- rspeer 9y agoThe machine can learn that someone named "Jamal" is more likely to be associated with the word "perpetrator", particularly in corpora centered on American news. This is likely a true fact about the world: one that results from racial profiling and unequal enforcement. It's not desirable to learn that, because encoding this in an AI system's belief about the "meaning" of the name "Jamal" will lead to more racial profiling. Just because something could be considered "true" doesn't mean it's good to design systems that will perpetuate it being true.
- mcguire 9y agoSuppose that the system "learned" that marriage consisted of one man and one (or very rarely more) woman. Would that be "reality"? (In fact, I'd rather bet that it did, and I congratulate the authors on the wisdom of not advertising that fact.) Various behavioral accidents can easily become embedded in culture, laws, and, yes, programs, at which point it stops mattering if they represent reality or "reality"; the real world will happily follow the cultural construction.