4 ms·

I don't know about this. I'm on board with not using race as a covariate -- people are born with their skin color and to penalize someone's loan application ove

I don't know about this. I'm on board with not using race as a covariate -- people are born with their skin color and to penalize someone's loan application over something they're born with and have no control over is unfair. But zip code? I understand there is empirical data showing how race is correlated with zip code, but you still choose your zipcode at the end of the day. If you have data saying that applicants from "Zipcode X" are less likely to repay loans, why is one's choice of zipcode out-of-bounds but their choice to have a hard credit check several months ago not? What if a poor credit history is correlated with race (fwiw I don't know if it is or not)? Is it then discriminatory to use credit history as a covariate? The truth is that machine learning to predict people's behavior is inherently discriminatory and involves generalizations by definition. Once you start disallowing training on covariates that exist due to voluntary behavior, it's not really clear whether you should be allowed to use machine learning to predict individual behavior at all (I'd probably listen to that argument actually). There's clearly a tradeoff to be made in terms of how much discrimination is allowable in exchange for a given unit of predictive performance, but that sounds incredibly difficult to regulate for every business's ML problem. I don't know what the answer is, but I think it's hasty to argue that anything that's correlated with race is out-of-bounds.

trowawee 8y ago
> I understand there is empirical data showing how race is correlated with zip code, but you still choose your zipcode at the end of the day. Unless you live in a city that was redlined[1], which is to say, unless you live in virtually any city. In which case, you really did not and in many cases still do not actually have much of a choice where you live. And that's without even looking at the fact that the racial wealth gap means that even if there weren't still strong barriers to integrated neighborhoods, lots of members of groups that were and are discriminated against would be unable to move into the neighborhoods that would theoretically let them escape that zip code bias. [1]: https://www.chicagomag.com/city-life/August-2017/How-Redlining-Segregated-Chicago-and-America/ https://www.chicagomag.com/city-life/August-2017/How-Redlini...
- deogeo 8y ago
  You make a good point, but you overstate the effects of the wealth gap. Going by absolute numbers, there are more white people living in poverty in the US than any other ethnicity [1], so you should be able to find a majority-white slum and move there. Of course, having to do so would be (is?) ridiculous. https://www.kff.org/other/state-indicator/poverty-rate-by-raceethnicity/?dataView=1&currentTimeframe=0&selectedRows=%7B%22wrapups%22:%7B%22united-states%22:%7B%7D%7D%7D&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D https://www.kff.org/other/state-indicator/poverty-rate-by-ra...
Yetanfou 8y ago
Because zip codes in themselves do not get assigned based on data relevant to whether someone is likely to pay off a loan, while credit do. Base decisions on data relevant to the matter at hand, not on unrelated data. As an example, some 'professional' sports are dominated by people of certain skin colours. Basketball is predominantly 'black', ice hockey predominantly 'white' while soccer is more representative of the population at large. If you're looking for people who have the potential to become good basketball players you might look at data on their athletic achievements, on their length and other such things. I don't think there is a need to have a dark skin to be able to play basketball at a high level even though the majority of players seem to have such. I assume scouts for the NBA, NHL and MLS don't profile people based on their skin colour but on the aforementioned things and others related to a person's ability to play the sport. [1] http://www.chineseorjapanese.com/wp-content/uploads/2009/04/trans0409raceinsports-b-1024x665.jpg http://www.chineseorjapanese.com/wp-content/uploads/2009/04/...

trowawee 8y ago

> I understand there is empirical data showing how race is correlated with zip code, but you still choose your zipcode at the end of the day. Unless you live in a city that was redlined[1], which is to say, unless you live in virtually any city. In which case, you really did not and in many cases still do not actually have much of a choice where you live. And that's without even looking at the fact that the racial wealth gap means that even if there weren't still strong barriers to integrated neighborhoods, lots of members of groups that were and are discriminated against would be unable to move into the neighborhoods that would theoretically let them escape that zip code bias. [1]: https://www.chicagomag.com/city-life/August-2017/How-Redlining-Segregated-Chicago-and-America/ https://www.chicagomag.com/city-life/August-2017/How-Redlini...

deogeo 8y ago
You make a good point, but you overstate the effects of the wealth gap. Going by absolute numbers, there are more white people living in poverty in the US than any other ethnicity [1], so you should be able to find a majority-white slum and move there. Of course, having to do so would be (is?) ridiculous. https://www.kff.org/other/state-indicator/poverty-rate-by-raceethnicity/?dataView=1&currentTimeframe=0&selectedRows=%7B%22wrapups%22:%7B%22united-states%22:%7B%7D%7D%7D&sortModel=%7B%22colId%22:%22Location%22,%22sort%22:%22asc%22%7D https://www.kff.org/other/state-indicator/poverty-rate-by-ra...

Yetanfou 8y ago

Because zip codes in themselves do not get assigned based on data relevant to whether someone is likely to pay off a loan, while credit do. Base decisions on data relevant to the matter at hand, not on unrelated data. As an example, some 'professional' sports are dominated by people of certain skin colours. Basketball is predominantly 'black', ice hockey predominantly 'white' while soccer is more representative of the population at large. If you're looking for people who have the potential to become good basketball players you might look at data on their athletic achievements, on their length and other such things. I don't think there is a need to have a dark skin to be able to play basketball at a high level even though the majority of players seem to have such. I assume scouts for the NBA, NHL and MLS don't profile people based on their skin colour but on the aforementioned things and others related to a person's ability to play the sport. [1] http://www.chineseorjapanese.com/wp-content/uploads/2009/04/trans0409raceinsports-b-1024x665.jpg http://www.chineseorjapanese.com/wp-content/uploads/2009/04/...