4 ms·
> If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices. Depends what
by wyager 10y ago
> If the training corpus for a machine learning model contains stereotypes and biases, then the output of the model will reflect those prejudices.
Depends what you mean by "the corpus". Let's take the example of loan applications. Maybe the training data contains credit scores, biographical information, and personal essays about the applicants from loan officers.
Let's say loan officers tend to use more negative language when talking about black people.
Any decent machine learning system given racial data will not adopt those biases. In fact, it will counteract them. It will determine that e.g. the semantic content of the essays is an indicator of loan suitability, and it will also notice if it has to e.g. add something to the essay score for black people to make the best decisions.
As long as the output data is objective, competent ML systems will account for any bias in the input data.
> Should we attempt to detect prejudices in machine learning models
We don't have to. If the output examples we are training on are true, the ML algorithm won't adopt any incorrect biases.
This is literally the point of machine learning. You want to make the best possible decision. If your algorithm has incorrect human biases, you're losing money. ML naturally accounts for this stuff.
- gugagore 10y agoYou are only talking about supervised learning. And the bias people are probably talking about is bias in the data, not bias in the "input", as you put it. See http://arxiv.org/abs/1607.06520 http://arxiv.org/abs/1607.06520 for a counterexample to your point.
- wyager 10y agoIn that paper it sounds like the algorithm is finding true relationships that the author doesn't like. What's the problem with the algorithm?
- gugagore 10y agoI don't understand what you mean by "the algorithm" or "true relationships". Do you know what I mean? There's truth as in what the data says, and there's truth as in what's underlying before social processes " corrupt " the truth. This paper examines ways to get at the latent truth. Do you think that's a useful endeavor? It's dismissive to say it has anything to do with what the "author likes".
- wyager 10y ago> and there's truth as in what's underlying before social processes " corrupt " the truth. Ah, so you're defining "truth" as "whatever I imagine the world would be like if society worked the way I wished it did". There's another word for that; "fantasy".
- gugagore 10y agoI asked you to define what you mean by "true relationship". And you did not.
- leftpad 10y ago> If the output examples we are training on are true, the ML algorithm won't adopt any incorrect biases. This is rarely the case when working wtih real data, and thus inspecting whether our models are biased against protected classes is probably one of the most important things an ML practitioner should do.
- wyager 10y agoWe have a good understanding of the error profiles of most ML algorithms. We can put tight bounds on the difference between predictions and validation data. If ML algorithms make mistakes, it's usually due to noise or low-quality data, not the algorithm itself. There is no reason an ML algorithm would be biased against a "protected class". It doesn't know what those are. It's possible that the algorithm will uncover a truth that you don't like, like that there as differences in risk profiles across race or gender, but that doesn't mean the algorithm is incorrectly biased. It just means that reality is at odds with how you might want it to be.
- leftpad 10y agoNot talking about the algorithm. It's the data that are biased. The choice of algorithm just determines how interpretable those biases actually are.
- wyager 10y ago> It's the data that are biased. Which data are you referring to? In most cases, the training data isn't human-generated, and if it is, we usually want to match human behavior as close as possible.
- leftpad 10y agoVirtually all data used to predict crimes or recidivism is fraught with human bias, for example. Not sure that we want to reproduce the bias of criminal justice system in any prediction problem involving this type of data. Read anything written by Solon Barocas: http://solon.barocas.org/ http://solon.barocas.org/