3 ms·

It's interesting to look at the actual course content here. https://ocw.mit.edu/courses/res-tll-008-social-and-ethical-responsibilities-of-computing-serc-fall-

by lacker 4y ago

It's interesting to look at the actual course content here. https://ocw.mit.edu/courses/res-tll-008-social-and-ethical-responsibilities-of-computing-serc-fall-2021/resources/res-tll008f21-6036_lab2/ https://ocw.mit.edu/courses/res-tll-008-social-and-ethical-r... More generally, what does it mean for a model to be "fair"? LIT Company’s Definition of Fairness (Group Unaware): The company believes that a fair process and, therefore, a fair model, would not account for gender or race at all. Advocacy Group's Definition (Demographic parity): An advocacy group believes that a model is fair if the distribution of outcomes for each demographic, gender, or other subgroup is the same among those that applied and those that were accepted. For example, in the example above, 30% of the applicants for loan applications come from women. In the demographic parity definition of fairness, this means 30% of the approved loan applications should come from women. I feel like the course content is somewhat slanted here. It is missing the definition of "fairness" in which you treat race and gender just like any other feature. Many systems work this way in practice - for example car insurance charges you differently by gender, because the statistics for genders are different. Ad-matching by gender and race is a longstanding practice. And any new system that you just train from scratch, by default it will not know to treat gender or race different from anything else. It is an interesting question, though. The main problems, I think, are practical ones - large enough AI models cannot be "race-blind" because if you remove race as a feature, they will be able to infer it anyways from proxy features. Whereas the only real way to enforce a system achieves the same percentage results for different groups is to add a "quota system" where you explicitly use different thresholds for different groups. So the practical alternatives often become "quota" or "nothing".

andersource 4y ago
Overall agree, although regarding > large enough AI models cannot be "race-blind" because if you remove race as a feature, they will be able to infer it anyways from proxy features In theory using a gradient reversal layer and an adversarial classifier you could do just that, to an extent. It could hurt the model's performance, which is exactly your point (should we ignore features with signal because they can be used to discriminate.)
wolverine876 4y ago
> It is missing the definition of "fairness" in which you treat race and gender just like any other feature. The real issue here, of course, is whether they are just like every other feature: Certainly in our society they are not perceived that way. People perceive very serious issues and have very strong feelings around race and gender. We see that right here on HN, of course. There is also, of course, a lot of discrimination by humans based on race and gender. If we want an unbiased, fair (and accurate) system, we have to correct for that. And the discrimination creates higher order effects: If there is discrimination against group X in K-12 education funding, then fewer of X will go to college, and fewer will have higher-paying jobs. If we then select blindly for income, we incorporate that bias (which might be appropriate if studying income by group, but not if we use it as a proxy for intelligence or effort). > the practical alternatives often become "quota" or "nothing". Those aren't practical alternatives, they are logical extremes creating a Manichean choice. Those are alternatives or a political debate, not for practical problem-solving.
RangerScience 4y ago
I'm pretty fascinated by all of this, although have barely dipped my toes in. IMHO - "Fairness" is a technical term-of-art meaning "the outcome _should not_ be effected by inputs X, Y and Z", and the collected science around making a system behave that way. It closely but not quite matches the colloquial meaning, ie "that's not fair!" - kinda like how "a fair coin" has a specific meaning that mostly tracks with how people use the word, but not quite, and with a lot more specificity. It's typically applied when you want to correct for real-world "unfair biases" in the training data; which in practical application is typically race, gender and the other legally protected categories - but AFAIK is just whichever inputs you decide you want to not have an impact on the outcome. AFAIK what you get out of the AI/ML "fairness" science is a way to measure, and correct for, dependency on the inputs that you (exterior to the system) have decided that you want to _not_ impact the outcome.
pfortuny 4y ago
The second definition assumes that the law of large numbers applies to any instance. It is impossible to satisfy each and every time. And it may also be blind to inherenet inequalities (as insurance companies know).
walnutclosefarm 4y ago
True story: at the height of fear about overrunning ICU facilities for Covid-19, a hospital for which a close relative works built a statistical model to predict which patients would benefit most from the ICU. The idea was that if there weren't sufficient beds, they'd triage so the beds went to those most likely to be saved from death, through the ICU interventions. The hospital's diversity committee wanted to modify the model so that a population-matching fraction of various minority groups were always selected (your quota). When told that the model didn't consider race, and that in order to achieve their desired goal, race would have to be added to the model, and artificial weights attached to race in order to bring things into balance (which would be a violation of Federal Law, BTW, as it directly and in a fully documented fashion discriminated in providing care based on race), the response was "That can't be true. If the model worked fairly races would be be treated equally." But it absolutely was true - because some minorities are far more likely to have, and to have untreated comorbidities that worsen outcomes for Covid-19, the model selected fewer of those minorities for the ICU. So, yeah, for the Diversity Committee definition of fairness, you need quotas.
gmac 4y ago
> for example car insurance charges you differently by gender, because the statistics for genders are different Interestingly, car insurers in the EU are no longer allowed to differentiate premiums by gender. But since this law came in, the difference between average male and female premiums has widened, based on correlated risk factors, such as occupation and model of car. https://www.theguardian.com/money/blog/2017/jan/14/eu-gender-ruling-car-insurance-inequality-worse https://www.theguardian.com/money/blog/2017/jan/14/eu-gender...

by lacker 4y ago

andersource 4y ago

Overall agree, although regarding > large enough AI models cannot be "race-blind" because if you remove race as a feature, they will be able to infer it anyways from proxy features In theory using a gradient reversal layer and an adversarial classifier you could do just that, to an extent. It could hurt the model's performance, which is exactly your point (should we ignore features with signal because they can be used to discriminate.)

wolverine876 4y ago

> It is missing the definition of "fairness" in which you treat race and gender just like any other feature. The real issue here, of course, is whether they are just like every other feature: Certainly in our society they are not perceived that way. People perceive very serious issues and have very strong feelings around race and gender. We see that right here on HN, of course. There is also, of course, a lot of discrimination by humans based on race and gender. If we want an unbiased, fair (and accurate) system, we have to correct for that. And the discrimination creates higher order effects: If there is discrimination against group X in K-12 education funding, then fewer of X will go to college, and fewer will have higher-paying jobs. If we then select blindly for income, we incorporate that bias (which might be appropriate if studying income by group, but not if we use it as a proxy for intelligence or effort). > the practical alternatives often become "quota" or "nothing". Those aren't practical alternatives, they are logical extremes creating a Manichean choice. Those are alternatives or a political debate, not for practical problem-solving.

RangerScience 4y ago

I'm pretty fascinated by all of this, although have barely dipped my toes in. IMHO - "Fairness" is a technical term-of-art meaning "the outcome _should not_ be effected by inputs X, Y and Z", and the collected science around making a system behave that way. It closely but not quite matches the colloquial meaning, ie "that's not fair!" - kinda like how "a fair coin" has a specific meaning that mostly tracks with how people use the word, but not quite, and with a lot more specificity. It's typically applied when you want to correct for real-world "unfair biases" in the training data; which in practical application is typically race, gender and the other legally protected categories - but AFAIK is just whichever inputs you decide you want to not have an impact on the outcome. AFAIK what you get out of the AI/ML "fairness" science is a way to measure, and correct for, dependency on the inputs that you (exterior to the system) have decided that you want to _not_ impact the outcome.

pfortuny 4y ago

The second definition assumes that the law of large numbers applies to any instance. It is impossible to satisfy each and every time. And it may also be blind to inherenet inequalities (as insurance companies know).

walnutclosefarm 4y ago

True story: at the height of fear about overrunning ICU facilities for Covid-19, a hospital for which a close relative works built a statistical model to predict which patients would benefit most from the ICU. The idea was that if there weren't sufficient beds, they'd triage so the beds went to those most likely to be saved from death, through the ICU interventions. The hospital's diversity committee wanted to modify the model so that a population-matching fraction of various minority groups were always selected (your quota). When told that the model didn't consider race, and that in order to achieve their desired goal, race would have to be added to the model, and artificial weights attached to race in order to bring things into balance (which would be a violation of Federal Law, BTW, as it directly and in a fully documented fashion discriminated in providing care based on race), the response was "That can't be true. If the model worked fairly races would be be treated equally." But it absolutely was true - because some minorities are far more likely to have, and to have untreated comorbidities that worsen outcomes for Covid-19, the model selected fewer of those minorities for the ICU. So, yeah, for the Diversity Committee definition of fairness, you need quotas.

gmac 4y ago

> for example car insurance charges you differently by gender, because the statistics for genders are different Interestingly, car insurers in the EU are no longer allowed to differentiate premiums by gender. But since this law came in, the difference between average male and female premiums has widened, based on correlated risk factors, such as occupation and model of car. https://www.theguardian.com/money/blog/2017/jan/14/eu-gender-ruling-car-insurance-inequality-worse https://www.theguardian.com/money/blog/2017/jan/14/eu-gender...