5 ms·
Biases in AI Systems
- deleted 5y ago[deleted]
- kingsuper20 5y agoThe oft-suppressed elephant in the room is 'what if the bias is correct?'. It's an uninteresting bug in the system up until that time.
- tr352 5y agoCan you given an example of a bias that is correct?
- nmca 5y agoLee Jussim has written some well-cited books on this topic, and makes some effort to handle it with care: https://en.m.wikipedia.org/wiki/Lee_Jussim https://en.m.wikipedia.org/wiki/Lee_Jussim
- hpoe 5y agoWell I feel that is a really really broad term to just ask for bias without really defining it but a couple off the top of my head are. 1. Someone from Utah is more likely to be a member of the Church of Jesus Christ of Latter Day Saints than someone from Pennsylvania. 2. Someone from an Arab speaking country is more likely to be Muslim than someone from a non Arab speaking country. 3. Someone who says "eh" at the end of every sentence is more likely to be Canadian. 4. Someone who says y'all is more likely to be from the south. 5. If someone asks me to "Please do the needful" they are likely from India. I've purposely chosen non extreme examples because there are many basis all over the place. Bais ≠ prejudice. Ultimately if we artificially restrain AI from being "baised" in any form we are really shooting ourselves and those most disadvantaged in the foot because instead of being able to use AI to discover the basis and then work on fixing it we instead just to pretend it doesn't exist. Finally a more provocative example. People who get pay day loans are less likely to pay back loans, black people are more likely to use pay day loans, ergo black people are more likely to default on loans. If we try and just force an AI to ignore this then we paper over the problem. If instead we start to examine causality we can start to figure out the root of the issue and how to address.
- commandlinefan 5y agoEven something as mundane as identifying a face as being "male" or "female" is fraught with controversy.
- samkater 5y agoI’m not outright disagreeing, but it seems your last statement contradicts the rest of the payday example. “If instead we start to examine causality we can start to figure out the root of the issue and how to address.” The causality piece is exactly the issue, right? People who use payday loans have less savings, more likely to work in jobs where their hours are unstable, have other poor financial indicators (past use of a payday loan, for example). Black people may disproportionately fall into this category, but I would argue it is wrong to effectively punish all black people (or conversely give other ethnicities an easier time) simply because of their race. Biases exist, no argument there. The dilemma is what we do with them.
- deleted 5y ago[deleted]
- username90 5y agoAnother way to word it is that an unbiased AI will never be able to perform better than humans at many tasks. Statistically accurate bias isn't a bug, it is a feature. Sometimes you want to avoid it for other reasons, like it feels wrong to assume traits are correlated with race etc, but by default the AI Should always be biased except for a few special cases.
- avs733 5y agoYou just described Phrenology.
- guythedudebro 5y agoNo they didn't.
- 5y ago
- alok-g 5y agoSee my comment here on definition of 'bias'. https://news.ycombinator.com/item?id=27631529 https://news.ycombinator.com/item?id=27631529
- jasonhong 5y agoThis is a really good question, one that we've discussed amongst at our university (a lot of researchers at Carnegie Mellon). To be useful, ML systems have to have some kind of bias. However, the distinction here is that some of these biases are harmful biases. Kate Crawford talks about allocative harms (how resources are allocated) and representation harms (e.g. stereotypes). Some of these harmful biases are really blatant. For example, labeling Blacks as "Gorillas" is offensive for many reasons. Some of these biases "correct" but that's due to biases in the data set or society. The ProPublica investigation of recidivism prediction is a good example, where it was more likely to say that Blacks should not be released. However, police are also more likely to arrest Blacks, which naturally leads to this bias. Other examples here include Amazon's resume system that was biased against women (since they used Amazon's hiring practices as ground truth), and image search for "professional hairstyles" that showed White women but "unprofessional hairstyles" that showed Black women. Other biases are also "correct" but greatly miss the underlying context. For example, a naive AI system might tell you don't go to a certain medical doctor that is a professor, since they have a higher rate of deaths. However, this doctor might also be a doctor of last resort, hence the high mortality rate. What I'm trying to get to is that even the term "correct" has a lot of subtleties to it. In many cases, figuring out what is "correct" (or ground truth in ML terms) can be a clash of values and world view, and might have different results based on differences in race, gender, age, culture, context, and power.
- kingsuper20 5y ago"Other examples here include Amazon's resume system that was biased against women (since they used Amazon's hiring practices as ground truth), and image search for "professional hairstyles" that showed White women but "unprofessional hairstyles" that showed Black women." Given enough data, and no doubt Amazon surveils their people more than most, they could determine the 'truth' along a more straightforward line. "Does this hair style make more money for the company" As hair can be a strong form of expression, there's probably a measurable delta here. Going forward, smart companies will obfuscate the determination. I suppose that training an AI is not a bad way to pull this off.
- LarryEt 5y agoHey yinz . My second home is PGH also although I am not there now. Such a thought provoking post. Thank you. So much to learn from this. I would expect no less from CMU.
- IfOnlyYouKnew 5y agoEven if a bias is “correct” at the population level, using it to make decisions regarding individuals is unjust and prone to be wrong. Example: men are known to commit the vast majority of violent crimes. But using that statistic to convict someone, deny them a job etc. would be inappropriate.
- ianhorn 5y agoRather than being a suppressed topic, in my experience, this is a case of people talking past each other. It's like correlation versus causation (versus plain old connected definitions). It can be true that A and B are correlated, while A doesn't cause B (or neither causes the other), and while their definitions have nothing to do with each other. Like nurse and gender. They're correlated in the US, but making someone a nurse doesn't change their gender, and the definitions have nothing to do with each other. Maybe in some countries the correlation is even flipped! Recall all the times in stats where an estimator can be an unbiased estimator of a correlation while being a biased estimator of a causal effect. So you get some people saying it (the correlation) is correct and other people saying it (the causal effect) is incorrect. Both are right! To stop talking past each other, they need to talk about bias with respect to the correlation or bias with respect to the causal effect in this particular direction. But what frustrates me is when the correlation side uses the (true) correlation to argue against a system being biased with regards to something else (w.r.t. a definition or w.r.t. a causal effect or w.r.t. a literal translation or w.r.t. some more complicated aspect of the system), and that harms are okay because the bias is a correct bias. We need to work on our terminology so that we can stop talking past each other. It doesn't help that our models have weird biases in absurdly complex function spaces, but we have to progress beyond a first-stats-course one-size-fits-all definition of bias.
- YeGoblynQueenne 5y agoThe phrase "the bias is correct" sounds like an oxymoron. Could you explain what you mean by it? Also, who is (oft-)suppressing the "elephant in the room"?
- bananabiscuit 5y agoI have a feeling you might already have a good hunch about the answers to your questions, but I’ll bite: If you see in a data set that Danes are tall, and that Kenyans are fast, and that Ashkenazi’s are smart, then it is a valid hypothesis that should not be thrown out outright, that the reason that’s the case is due to actual differences inherent to the population groups and not any other confounding factors. As for your second question: mostly progressives, leftists and liberals.
- deleted 5y ago[deleted]
- YeGoblynQueenne 5y agoPlease don't bite unless you're a fish. I'm certainly not trying to reel you in. Rather the reason I'm asking is that the expression you used is vague and imprecise and I don't know what exactly you mean by it. As you suspect, I can guess what you might mean, but if we start double-guessing each other, we're just injecting noise in the conversation and a few comments from now we end up completely confused about what each other is trying to say. So much better to establish some common language before we waste time talking past each other. Doesn't that make sense? Keeping that in mind, I am still not happy I understand what you mean with the following: >> If you see in a data set that Danes are tall, and that Kenyans are fast, and that Ashkenazi’s are smart, then it is a valid hypothesis that should not be thrown out outright, that the reason that’s the case is due to actual differences inherent to the population groups and not any other confounding factors. The reason for my continued uncertainty is that you do not say, in the above example, what is the "bias" and how it is "correct". You've identified a hypothesis that you can make about the data (rather than a hypothesis derived from the data, i.e. some kind of model that explains the data). But a hypothesis is not bias. A hypothesis can be correct or incorrect, and bias may play a part in that, or not. But a hypothesis is a hypothesis and bias is bias. So what is "bias", the way you mean it? >> As for your second question: mostly progressives, leftists and liberals. Ugh. I shouldn't have asked. I'm going to take a wild guess that you're from the USA and that you have some kind of stake at the culture war you folks got brewing over there. I'm not from over there and I want no part in that. So forget I asked. And good luck getting all that sorted out between you.
- LarryEt 5y agoI just started on Kahneman's Noise: A Flaw in Human Judgment 2 days ago. I mean it took so long for Kahneman and Tversky ideas on bias to disperse that we can even be talking about bias in this context. Bias isn't even the real problem with ML, noise is obviously.
- visarga 5y agoA very good paper.