3 ms·

Presumably black people who are great loan customers would be incorrectly refused by a system based on fashion or addresses alone, so your answer seems to be th

Presumably black people who are great loan customers would be incorrectly refused by a system based on fashion or addresses alone, so your answer seems to be that yes, racist methods actually lose the bank money. My tendency is to think that since it is optimized to maximize profit, the more features about a person we get in the data, the less "racist" the system becomes under this definition. Yes it's more possible to "hide" racism in a complex method using tons of features, but increasingly less likely it would happen. If we can use income and debt and whatever other things to make either a good classifier that ignores race, or an inferior one that sneakily identifies race and bases the decision on that, mathematical optimization of the model should result in the former. ML can be trusted more than humans in this situation, not less. Of course there's still the issue of the data being biased, which is where it all started.

mlyle 6y ago
> Presumably black people who are great loan customers would be incorrectly refused by a system based on fashion or addresses alone, Strawman. The question is whether predictions are improved by using race (or direct proxies for race), not whether it'd be wise to use proxies for race alone. Loan default rates are higher for disadvantaged minorities, even after controlling for many, many other variables (income, neighborhood, level of education, etc). Therefore, using race (or inferring race) improves prediction quality, but is ethically dubious.
- unishark 6y ago
  > Strawman. Can we skip this kind of snippy arguing please? Anyway I take it you agree the statement is true but don't want to. So now you're saying those great minority loan customers are impossible to identify? I think you just need to figure out what information is still missing. What's the effect size at this point anyway?
  - mlyle 6y ago
    > > Strawman. > Can we skip this kind of snippy arguing please? If you don't start off by willfully mischaracterizing your opponent's argument in order to be able to more easily refute it, I think you'll find that they accuse you of this less. I think you're just willfully missing the point. Ideally, you'd just lend money to the people who will pay you back. Unfortunately, we can't predict this perfectly. Adding race, or proxies for race, to the things you consider improves your predictions somewhat.
    - unishark 6y ago
      Nope it was a completely honest question and what I considered a constructive line of thought. You should assume good faith. I asked about two competing definitions of what is racist and you seemed to prefer one over the other. And what have I claimed to "easily refute" exactly? More like I ran with the definition, and considered how to address the problem as stated. I said more features were needed, I didn't say enough features were currently used. You keep pointing to a dichotomy between perfect and flawed, while I was talking about relative improvements. There isn't even necessarily a disagreement there.
      - mlyle 6y ago
        I sez, "other variables + race makes better predictions of default risk than other variables alone". U sez, "LOL -only- using race will cost banks money" I sez, "???"
jay_kyburz 6y ago
I wonder what a court would rule if you went through your algorithm line by line and demonstrated that is was only designed to maximize profit, and the fact that it was racist was only incidental. I'm guessing it's illegal to discriminate based on race, gender or ability no matter how much it extra it costs a business, whether its deliberate or not.
- unishark 6y ago
  Well in places like NYC the laws are strict regarding rental housing. Landlords need to establish consistent requirements then accept the first applicant that fits them. Typically they require the income to rent ratio is greater than three or something like that, which undoubtedly has a disparate effect when it comes to people of different races given how high the rents are and how incomes differ. Are you calling this incidental racism?
danielheath 6y ago
The issue is that "fashion for minorities" is correlated with being a minority which is correlated with poverty which is correlated with high credit risk. This makes "buys fashion for minorities" quite an accurate predictor of high credit risk, but also a profoundly racist one.
- unishark 6y ago
  Yes I got that. The question is whether simply discarding the overtly-race-based features from the data will be enough.

mlyle 6y ago

> Presumably black people who are great loan customers would be incorrectly refused by a system based on fashion or addresses alone, Strawman. The question is whether predictions are improved by using race (or direct proxies for race), not whether it'd be wise to use proxies for race alone. Loan default rates are higher for disadvantaged minorities, even after controlling for many, many other variables (income, neighborhood, level of education, etc). Therefore, using race (or inferring race) improves prediction quality, but is ethically dubious.

unishark 6y ago
> Strawman. Can we skip this kind of snippy arguing please? Anyway I take it you agree the statement is true but don't want to. So now you're saying those great minority loan customers are impossible to identify? I think you just need to figure out what information is still missing. What's the effect size at this point anyway?
- mlyle 6y ago
  > > Strawman. > Can we skip this kind of snippy arguing please? If you don't start off by willfully mischaracterizing your opponent's argument in order to be able to more easily refute it, I think you'll find that they accuse you of this less. I think you're just willfully missing the point. Ideally, you'd just lend money to the people who will pay you back. Unfortunately, we can't predict this perfectly. Adding race, or proxies for race, to the things you consider improves your predictions somewhat.
  - unishark 6y ago
    Nope it was a completely honest question and what I considered a constructive line of thought. You should assume good faith. I asked about two competing definitions of what is racist and you seemed to prefer one over the other. And what have I claimed to "easily refute" exactly? More like I ran with the definition, and considered how to address the problem as stated. I said more features were needed, I didn't say enough features were currently used. You keep pointing to a dichotomy between perfect and flawed, while I was talking about relative improvements. There isn't even necessarily a disagreement there.
    - mlyle 6y ago
      I sez, "other variables + race makes better predictions of default risk than other variables alone". U sez, "LOL -only- using race will cost banks money" I sez, "???"

jay_kyburz 6y ago

I wonder what a court would rule if you went through your algorithm line by line and demonstrated that is was only designed to maximize profit, and the fact that it was racist was only incidental. I'm guessing it's illegal to discriminate based on race, gender or ability no matter how much it extra it costs a business, whether its deliberate or not.

unishark 6y ago
Well in places like NYC the laws are strict regarding rental housing. Landlords need to establish consistent requirements then accept the first applicant that fits them. Typically they require the income to rent ratio is greater than three or something like that, which undoubtedly has a disparate effect when it comes to people of different races given how high the rents are and how incomes differ. Are you calling this incidental racism?

danielheath 6y ago

The issue is that "fashion for minorities" is correlated with being a minority which is correlated with poverty which is correlated with high credit risk. This makes "buys fashion for minorities" quite an accurate predictor of high credit risk, but also a profoundly racist one.

unishark 6y ago
Yes I got that. The question is whether simply discarding the overtly-race-based features from the data will be enough.