4 ms·

> If you want to claim you know what fair is in any given situation, then go and hardcode all your own fair rules This strikes me as eerily similar to the argu

> If you want to claim you know what fair is in any given situation, then go and hardcode all your own fair rules This strikes me as eerily similar to the argument that type systems are impossible because of the halting problem. It's sort of true in some sense, but not in an even remotely useful way. So it mostly functions as a way of derailing the conversation away of the more subtle distinctions that do matter (e.g., could we design an easy-to-use type system that rules out this particular type of non-termination/other class of bugs). There's a large middle-ground between "hard code all your own rules" and "completely unconstrained learning". Learning under constraints is not a new idea. A classical programming analogy to your argument might be "well the halting problem is undecidable so ignore all this high-level language stuff and just go code up your own turing machine; it's the best you'll ever be able to do". > because you aren't going to find "fairness" in machine learning. Why not? The human notion of fairness is fuzzy, which is why researchers have provided various formal notions of fairness in machine learning tasks. Obviously, these formal definitions may or may not correspond to your own gut instinct about what is "fair". And there might be friction between different notions of fairness. None of that should be surprising; otherwise, fairness wouldn't be something that philosophers continue to bleed ink about. But it is equally obvious that for some notions of fairness, there will exist machine learning algorithms that learn well under the given constraint.

kyleperik 8y ago
I agree with you when you say fairness is fuzzy. Though this implies that there are no set of rules that define it, that also means there is no way to train on it. By choosing the right features and utilizing it in the right way I believe you can avoid putting people in bad situations for bad reasons. I'm saying, if you train an algorithm to guess if someone is involved with crime, it's going to be incredibly stereotypical with it's answers. Same as if you rely completely on statistics. I'm not actually saying hardcore all your rules. I was making an example that if you really know exactly what fairness is, then program it. But we both don't know there are no definite rules. So why hardcode at all? I think it is never the ML that is "unfair", it's the one who made it who is responsible. If you're finding yourself running into issues in your ML with unfairness, I think you're just using it wrong. EDIT: Rewording last sentence
romaniv 8y ago
The problem here is that modern machine learning is rarely more than a computerized way of building statistical models based on observed data. And the bias mitigation techniques I've read about all revolve around (indirectly) manipulating those models based on what results they produce. This can lead to various problems in the long run.
commandlinefan 8y ago
> The human notion of fairness is fuzzy ... to the point that every attempt to define "fairness" in one (arguable) aspect leads to (codified, inescapable) unfairness in another.

kyleperik 8y ago

I agree with you when you say fairness is fuzzy. Though this implies that there are no set of rules that define it, that also means there is no way to train on it. By choosing the right features and utilizing it in the right way I believe you can avoid putting people in bad situations for bad reasons. I'm saying, if you train an algorithm to guess if someone is involved with crime, it's going to be incredibly stereotypical with it's answers. Same as if you rely completely on statistics. I'm not actually saying hardcore all your rules. I was making an example that if you really know exactly what fairness is, then program it. But we both don't know there are no definite rules. So why hardcode at all? I think it is never the ML that is "unfair", it's the one who made it who is responsible. If you're finding yourself running into issues in your ML with unfairness, I think you're just using it wrong. EDIT: Rewording last sentence

romaniv 8y ago

The problem here is that modern machine learning is rarely more than a computerized way of building statistical models based on observed data. And the bias mitigation techniques I've read about all revolve around (indirectly) manipulating those models based on what results they produce. This can lead to various problems in the long run.

commandlinefan 8y ago

> The human notion of fairness is fuzzy ... to the point that every attempt to define "fairness" in one (arguable) aspect leads to (codified, inescapable) unfairness in another.