3 ms·

Since computers are unbiased, the idea is that machine learning is also unbiased. However, that's not true and, if anything, it actually re-enforces a lot of b

Since computers are unbiased, the idea is that machine learning is also unbiased. However, that's not true and, if anything, it actually re-enforces a lot of bias. I've never done any, but my understanding is that machine learning is just correlation. It's good at figuring out "what", but not "why". Consider training an algorithm to recognize horses by feeding it millions of pictures of horses. Eventually, the algorithm "learns" what a horse is, but it's definition of a horse is based on the inputs it was given by a human. So now, consider the scenario where the millions of pictures of horses were all brown. If you give the algorithm a picture of a white horse, it'll tell you it's not a horse. If you give it a picture of a brown donkey, it might think it's a horse because it's learned to put too much emphasis on the color brown. If that algorithm becomes relied on to define a horse, "the system" will insist there are no white horses even though you can walk outside and see them plain as day. Now, apply the same kind of idea and feed an algorithm mugshots of all criminals. It's going to develop the same bias and tell you that a black person is more likely to be a criminal than a white person. There's no nuance. The inputs used to train the AI were tainted by decades of systematic discrimination, but the AI doesn't know that. Of course you could try to take that input bias into account, but the whole sales pitch of machine learning is that you feed it tons of data and it gives you an objective result. As far as I know, no one is trying to quantify, and correct, the biases in the inputs. The phrase "money laundering for bias" means the machine learning algorithms are used to re-enforce incorrect opinions and assumptions because it gives the excuse that an "objective" computer used cold hard data to draw the same conclusion. Machine learning is one of the scariest parts of tech right now because it's the equivalent of an extremely stupid person that only understands correlation and not causality and the systems being built are going to be making a lot of decisions at scale.

vngzs 6y ago
I share your concerns. I am of the opinion that we shouldn't be using machine learning algorithms to make decisions of consequence in people's lives without filtering those decisions through human operators. You can use machine learning to assist these human operators and empower them to do much more work than without the technology. But at the end of the day, you still have someone to hold responsible, and you still have checks on the ML doing something profoundly unethical.
- donmcronald 6y ago
  > You can use machine learning to assist these human operators and empower them to do much more work than without the technology. But at the end of the day, you still have someone to hold responsible, and you still have checks on the ML doing something profoundly unethical. I would, but the people in charge won't. Some of my family members MUST chat with a crappy bot before they can get support from their mobile phone provider and that's in Canada where we pay an astronomical amount of money for our phones/plans. A penny saved is a penny earned, even if it costs someone else a dollar.
dmurray 6y ago
> Now, apply the same kind of idea and feed an algorithm mugshots of all criminals. It's going to develop the same bias and tell you that a black person is more likely to be a criminal than a white person. No. Nobody training an ML system to detect criminals would train it only with pictures of criminals. And if you somehow did, it wouldn't determine that black people are more likely to be criminals, but that humans are more likely to be criminals than say ducks or fire engines. Yes, ML models can end up reflecting prejudices in their training data, but this description is incorrect, reductionist and unhelpful.
- donmcronald 6y ago
  How would you even start to build a ML training dataset that isn't biased by the discriminatory policing the US has seen for the last 50 years?
nmfisher 6y ago
The problem is we actually conflate two related, but not identical, concepts when we talk about "bias". The first is statistical bias - feeding algorithms training data that is somehow unrepresentative of the "real world" (or more specifically, the actual class of data for the intended use case). As an example, applying facial recognition to Caucasian faces when the model was solely trained on Chinese faces, you're going to have a bad time. The problem was that your data was "biased", because you actually wanted a model that recognizes "human faces", but you trained on the biased subset of "Chinese faces". The second is the ethical/political notion of "bias" against individuals; more concretely, the idea that a society is "just" when people are judged as individuals, and not prejudiced by their gender/skin colour/etc. In this respect, when we say "we should not be biased against men", we really mean that "an individual man should not be treated any differently from a woman, even though men are overwhelmingly perpetrators (and victims) homicide". The complication is that reality is inherently imbalanced/biased. Society can be chopped up into a lot of sub-views that skew towards particular demographics. Some are relatively harmless. "OnlyFans content creators" aren't 50-50 men-women, and men aren't charging the same as women either. Some are not - "murderers" are mostly men, black men are overrepresented in the "criminal" group, and so on. This raises some obvious questions: 1) Why is this the case? Is this the result of systemic discrimination? Historical oppression? Innate preference? Cultural pressure to conform? 2) If you can answer (1), how does that influence your view of what a "just" society is? For example, do you consider it to be "unjust" to be wary of men (and men only) to protect yourself from random physical violence when you're out and about? 3) Does everyone share your view on what a "just" society is? 4) How do these answers dictate what you should "do" about it? As a voter? As an ML practitioner? As a CEO? I'm not going to delve further into these questions, because they cause a lot of contention and deserve more time/consideration than I can justify right now in a HN post. The main reason I decided to comment is that I've seen too much debate that tries to steamroll people into accepting conclusions without considering or answering these questions. Even worse, some people are actively trying to silence others who simply want to discuss these questions, rather than swallowing their conclusions uncritically. (This is not levelled at you, by the way, your comment just presented an opportunity to lay out my thoughts.) I don't think anyone can meaningfully discuss the (ethical) concept of "bias" without first laying out a very comprehensive perspective of "society" that touches on all of these points. There was a joint paper from Google, Facebook (and possibly others) about 2 years ago that I thought handled this exceptionally well. The authors addressed many of these questions honestly and objectively, and most importantly, acknowledged the potential for disagreement.

vngzs 6y ago

I share your concerns. I am of the opinion that we shouldn't be using machine learning algorithms to make decisions of consequence in people's lives without filtering those decisions through human operators. You can use machine learning to assist these human operators and empower them to do much more work than without the technology. But at the end of the day, you still have someone to hold responsible, and you still have checks on the ML doing something profoundly unethical.

donmcronald 6y ago
> You can use machine learning to assist these human operators and empower them to do much more work than without the technology. But at the end of the day, you still have someone to hold responsible, and you still have checks on the ML doing something profoundly unethical. I would, but the people in charge won't. Some of my family members MUST chat with a crappy bot before they can get support from their mobile phone provider and that's in Canada where we pay an astronomical amount of money for our phones/plans. A penny saved is a penny earned, even if it costs someone else a dollar.

dmurray 6y ago

> Now, apply the same kind of idea and feed an algorithm mugshots of all criminals. It's going to develop the same bias and tell you that a black person is more likely to be a criminal than a white person. No. Nobody training an ML system to detect criminals would train it only with pictures of criminals. And if you somehow did, it wouldn't determine that black people are more likely to be criminals, but that humans are more likely to be criminals than say ducks or fire engines. Yes, ML models can end up reflecting prejudices in their training data, but this description is incorrect, reductionist and unhelpful.

donmcronald 6y ago
How would you even start to build a ML training dataset that isn't biased by the discriminatory policing the US has seen for the last 50 years?

nmfisher 6y ago

The problem is we actually conflate two related, but not identical, concepts when we talk about "bias". The first is statistical bias - feeding algorithms training data that is somehow unrepresentative of the "real world" (or more specifically, the actual class of data for the intended use case). As an example, applying facial recognition to Caucasian faces when the model was solely trained on Chinese faces, you're going to have a bad time. The problem was that your data was "biased", because you actually wanted a model that recognizes "human faces", but you trained on the biased subset of "Chinese faces". The second is the ethical/political notion of "bias" against individuals; more concretely, the idea that a society is "just" when people are judged as individuals, and not prejudiced by their gender/skin colour/etc. In this respect, when we say "we should not be biased against men", we really mean that "an individual man should not be treated any differently from a woman, even though men are overwhelmingly perpetrators (and victims) homicide". The complication is that reality is inherently imbalanced/biased. Society can be chopped up into a lot of sub-views that skew towards particular demographics. Some are relatively harmless. "OnlyFans content creators" aren't 50-50 men-women, and men aren't charging the same as women either. Some are not - "murderers" are mostly men, black men are overrepresented in the "criminal" group, and so on. This raises some obvious questions: 1) Why is this the case? Is this the result of systemic discrimination? Historical oppression? Innate preference? Cultural pressure to conform? 2) If you can answer (1), how does that influence your view of what a "just" society is? For example, do you consider it to be "unjust" to be wary of men (and men only) to protect yourself from random physical violence when you're out and about? 3) Does everyone share your view on what a "just" society is? 4) How do these answers dictate what you should "do" about it? As a voter? As an ML practitioner? As a CEO? I'm not going to delve further into these questions, because they cause a lot of contention and deserve more time/consideration than I can justify right now in a HN post. The main reason I decided to comment is that I've seen too much debate that tries to steamroll people into accepting conclusions without considering or answering these questions. Even worse, some people are actively trying to silence others who simply want to discuss these questions, rather than swallowing their conclusions uncritically. (This is not levelled at you, by the way, your comment just presented an opportunity to lay out my thoughts.) I don't think anyone can meaningfully discuss the (ethical) concept of "bias" without first laying out a very comprehensive perspective of "society" that touches on all of these points. There was a joint paper from Google, Facebook (and possibly others) about 2 years ago that I thought handled this exceptionally well. The authors addressed many of these questions honestly and objectively, and most importantly, acknowledged the potential for disagreement.