3 ms·

> The whole point of white listing variables is to only allow things you are comfortable using todiscimrinate. The problem is that variables where you can safe

> The whole point of white listing variables is to only allow things you are comfortable using todiscimrinate. The problem is that variables where you can safely discriminate don't exist in the real world because everything is intertwined. For example, if you see that somebody went to school at {Howard, Bryn Mawr, BYU} you can make some pretty solid inferences about their {race, gender, religion} even if they don't explicitly provide them.

pclmulqdq 3y ago
These schoools' associations are all correlations, not guarantees, though. Howard is a historically black college, which means their student body today is majority black, not 100% black. Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life. BYU is a Christian school, but also admits several non-religious people too, and people regularly become apostates to their respective religion.
- Wowfunhappy 3y ago
  > Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life. Technically true of course, but >99% of the population will identify with a single gender throughout their life. The correlation is so strong that it's not even really a correlation anymore.
  - pclmulqdq 3y ago
    If you're going with a probabilistic argument, I think BYU students are more likely to stick with mormonism (let alone Christianity) than Bryn Mawr students with their gender. That doesn't mean you aren't stereotyping people, though.
    - Wowfunhappy 3y ago
      Irrespective of the AI discussion (computers don't care if you're stereotyping or not), I really don't think it's stereotyping to assume someone who went to a school which literally only admits women is probably a woman. There's a difference between assuming someone's gender because they're a cheerleader, and assuming because they won the women's cheerleading championship. I also assume that the person who drives a fire truck is a fire fighter.
- michaelt 3y ago
  Right, but the fact the associations are only correlations won't stop the ML model from learning them, if they're present in the training data. If the training data shows a history of bias against Howard graduates, the model will learn to replicate.
jncfhnb 3y ago
No, again, it is inevitable that outcomes will not be equitable across race. That doesn’t mean all variables are unjustified to use. Job applications are kind of dumb. The common use case is credit decisions. Surely, for example, you’re ok looking at credit history and income when deciding to make a loan, right? Even though it won’t be equitable outcomes across races, it is still “fair”
- parpfish 3y ago
  The goal of fairness ML models is not necessarily to have equal outcomes across subgroups. Other goals can be to have equal model accuracy across subgroups (e.g., models has the same precision/auroc/etc for each subgroup, even if the predictions themselves differ across subgroups) or identical distribution of statistical biases across subgroups (e.g., the model makes the same types of mistakes within each subgroup)
  - jncfhnb 3y ago
    While a lot of people will state those are appropriate goals, they are not. They’re really dumb goals. The model being equally bad for each group is not an indicator that you have achieved anything. Most likely you’ve just made your model for white people worse. Like, let’s look at that for credit decisions. Black folks have a lower income distribution. So it’s like saying the model is ok if it rejects a Black applicant making 80k if it would also reject a White candidate making 100k. That is the insane bullshit you’re effectively supporting by desiring comparable error distributions. You have to accept that equal outcomes are not just impossible but not desirable. And then you only use fields that people agree are fair to use. If Black people get disproportionately better or worse outcomes it is either because those outcomes make sense given their stats or it’s not ok because if a White dude had the same info, that you agree is fair to use, they would have passed and the underlying reason for the difference is some bullshit other input. That might mean that the model is a dumb: you must make $100k for this loan (illustrative) which is going to mean a lot of Black folks cannot get it. Well that model is at least “fair”. Accommodations for race should not be coming from an AI model
    - 3y ago

pclmulqdq 3y ago

These schoools' associations are all correlations, not guarantees, though. Howard is a historically black college, which means their student body today is majority black, not 100% black. Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life. BYU is a Christian school, but also admits several non-religious people too, and people regularly become apostates to their respective religion.

Wowfunhappy 3y ago
> Bryn Mawr is a womens' school and admits only women. However, gender is not always fixed going through life. Technically true of course, but >99% of the population will identify with a single gender throughout their life. The correlation is so strong that it's not even really a correlation anymore.
- pclmulqdq 3y ago
  If you're going with a probabilistic argument, I think BYU students are more likely to stick with mormonism (let alone Christianity) than Bryn Mawr students with their gender. That doesn't mean you aren't stereotyping people, though.
  - Wowfunhappy 3y ago
    Irrespective of the AI discussion (computers don't care if you're stereotyping or not), I really don't think it's stereotyping to assume someone who went to a school which literally only admits women is probably a woman. There's a difference between assuming someone's gender because they're a cheerleader, and assuming because they won the women's cheerleading championship. I also assume that the person who drives a fire truck is a fire fighter.
michaelt 3y ago
Right, but the fact the associations are only correlations won't stop the ML model from learning them, if they're present in the training data. If the training data shows a history of bias against Howard graduates, the model will learn to replicate.

jncfhnb 3y ago

No, again, it is inevitable that outcomes will not be equitable across race. That doesn’t mean all variables are unjustified to use. Job applications are kind of dumb. The common use case is credit decisions. Surely, for example, you’re ok looking at credit history and income when deciding to make a loan, right? Even though it won’t be equitable outcomes across races, it is still “fair”

parpfish 3y ago
The goal of fairness ML models is not necessarily to have equal outcomes across subgroups. Other goals can be to have equal model accuracy across subgroups (e.g., models has the same precision/auroc/etc for each subgroup, even if the predictions themselves differ across subgroups) or identical distribution of statistical biases across subgroups (e.g., the model makes the same types of mistakes within each subgroup)
- jncfhnb 3y ago
  While a lot of people will state those are appropriate goals, they are not. They’re really dumb goals. The model being equally bad for each group is not an indicator that you have achieved anything. Most likely you’ve just made your model for white people worse. Like, let’s look at that for credit decisions. Black folks have a lower income distribution. So it’s like saying the model is ok if it rejects a Black applicant making 80k if it would also reject a White candidate making 100k. That is the insane bullshit you’re effectively supporting by desiring comparable error distributions. You have to accept that equal outcomes are not just impossible but not desirable. And then you only use fields that people agree are fair to use. If Black people get disproportionately better or worse outcomes it is either because those outcomes make sense given their stats or it’s not ok because if a White dude had the same info, that you agree is fair to use, they would have passed and the underlying reason for the difference is some bullshit other input. That might mean that the model is a dumb: you must make $100k for this loan (illustrative) which is going to mean a lot of Black folks cannot get it. Well that model is at least “fair”. Accommodations for race should not be coming from an AI model
  - 3y ago