3 ms·

I think "unfair" implies prejudice and bias. Do you take it as axiomatic that a system that produces different outcome for different racial groups is "unfair"?

j2kun 11y ago
This is precisely the question :) we just don't know what the right definitions are, and neither do we have a good understanding of the consequences of existing attempts.
- murbard2 11y ago
  You don't seem to be asking that question at all. You seem to be including disparate outcome in the set of "unfair" things, by fiat, and from there looking for the logical closure of that concept under machine learning. That closure is "Harrison Bergeron"... but one man's modus ponens is another's modus tollens.
  - j2kun 11y ago
    I think I'm quite clear: we don't have an accepted and useful mathematical definition of what fairness means. This should, at least on some level, line up with our society's legal standards (as the corresponding mathematical definitions of privacy, security, etc. do) while allowing us to prove theorems or at least make well-defined conjectures about the properties of algorithms. The question is what counts as fair. I'm not saying disparate outcomes is necessary or sufficient, though it receives a lot of attention. Even if we could quantify which disparate outcomes are adverse, it's not obvious how to design algorithms that meet those criteria.
lutorm 11y ago
It seems a system that produces different outcomes for two individuals differing only in their racial identity, or other "irrelevant factors", is unfair. The question is, of course, what factors actually are relevant.
- msandford 11y ago
  > It seems a system that produces different outcomes for two individuals differing only in their racial identity, or other "irrelevant factors", is unfair. The question is, of course, what factors actually are relevant. I'm interested in a little thought experiment: 1. for the sake of this algorithm there are two people: a white person and black person 2. race and other "irrelevant factors" aren't recorded or used 3. these people are except for race, otherwise identical 4. how can the algorithm discriminate between them? It seems tautological that lacking race as a feature to discriminate with the algorithm won't be able to successfully discriminate, right? So how does the discriminatory information somehow bleed into the system such that it negatively affects two otherwise identical (or practically so) people? I'm not trying to defend the algorithm as the perfect arbiter or anything. I'm just honestly a little confused. Is it that the irrelevant factors are actually recorded somehow, but shouldn't be?
- murbard2 11y ago
  The author explicitly acknowledges that relevant factors correlate, in fact he suggests race as a factor will creep back in even if it is excluded as an independent variable. He is thus suggesting that even given genuine group differences, different group outcomes are intrinsically unfair if the grouping happens to be race.

j2kun 11y ago

This is precisely the question :) we just don't know what the right definitions are, and neither do we have a good understanding of the consequences of existing attempts.

murbard2 11y ago
You don't seem to be asking that question at all. You seem to be including disparate outcome in the set of "unfair" things, by fiat, and from there looking for the logical closure of that concept under machine learning. That closure is "Harrison Bergeron"... but one man's modus ponens is another's modus tollens.
- j2kun 11y ago
  I think I'm quite clear: we don't have an accepted and useful mathematical definition of what fairness means. This should, at least on some level, line up with our society's legal standards (as the corresponding mathematical definitions of privacy, security, etc. do) while allowing us to prove theorems or at least make well-defined conjectures about the properties of algorithms. The question is what counts as fair. I'm not saying disparate outcomes is necessary or sufficient, though it receives a lot of attention. Even if we could quantify which disparate outcomes are adverse, it's not obvious how to design algorithms that meet those criteria.

lutorm 11y ago

It seems a system that produces different outcomes for two individuals differing only in their racial identity, or other "irrelevant factors", is unfair. The question is, of course, what factors actually are relevant.

msandford 11y ago
> It seems a system that produces different outcomes for two individuals differing only in their racial identity, or other "irrelevant factors", is unfair. The question is, of course, what factors actually are relevant. I'm interested in a little thought experiment: 1. for the sake of this algorithm there are two people: a white person and black person 2. race and other "irrelevant factors" aren't recorded or used 3. these people are except for race, otherwise identical 4. how can the algorithm discriminate between them? It seems tautological that lacking race as a feature to discriminate with the algorithm won't be able to successfully discriminate, right? So how does the discriminatory information somehow bleed into the system such that it negatively affects two otherwise identical (or practically so) people? I'm not trying to defend the algorithm as the perfect arbiter or anything. I'm just honestly a little confused. Is it that the irrelevant factors are actually recorded somehow, but shouldn't be?
murbard2 11y ago
The author explicitly acknowledges that relevant factors correlate, in fact he suggests race as a factor will creep back in even if it is excluded as an independent variable. He is thus suggesting that even given genuine group differences, different group outcomes are intrinsically unfair if the grouping happens to be race.