3 ms·

You’re missing the broader picture. This is not a problem with data, AI, algorithms or whatever. This is simply a mathematical issue that is fundamentally unsol

by tomp 6y ago

You’re missing the broader picture. This is not a problem with data, AI, algorithms or whatever. This is simply a mathematical issue that is fundamentally unsolvable. Imagine you have two groups of people that have different means and distributions of “input” (could be intelligence, wealth, interests, ...), and this input is somehow correlated with some output (insurance rate, criminality, university acceptance rate). Both input and output are one-dimensional. Then you only have two choices. (1) You “unify” the distribution and use it as joint input, resulting in one group (the one with the higher mean) getting “more” - many people think this is unfair. (2) Or you can subtract the means first and so make the distributions as equal as possible, which can result in a person from the “more” group having worse outcome than an equivalent person from the “less” group (if the means are close together and the distributions are wide, there will be many such people) - many people would consider this unfair. There are plenty of examples, current and historical, of society picking one or the other, for different groups and topics: wealth/success/income (old vs young, Jews&Asians vs other Americans, men vs women, whites vs blacks), university acceptance (Jews used to be discriminated against (look up “Jewish problems”), now Asians are, men vs women [1], “positive” discrimination of blacks) - just a few. In many examples historically, we consider (1) fundamentally better than (2) - e.g. discrimination against Jews at universities - but it seems it takes the society a long time to reach this conclusion in each instance. [1] https://en.wikipedia.org/wiki/Simpson%27s_paradox#UC_Berkeley_gender_bias https://en.wikipedia.org/wiki/Simpson%27s_paradox#UC_Berkele...

mlyle 6y ago
No, indeed, I think that's the whole point. Approach #1 is to just go to maximize expected value, using all variables, including race. Approach #2 is to use race to adjust the distributions as you suggest. Call this "affirmative action." The inbetween approach #1.5, that we usually follow, is to make a value judgment about whether any individual item is OK to use. Income? Of course it's reasonable to use income to determine whether to lend-- even if it's correlated with race. Race? No, that seems wrong. Location? Sure--- wait, you've drawn a box around all the black neighborhoods, nope! Buys certain products? Sure-- wait, you don't mean "buys products that blacks stereotypically like," do you? #1.5 is already iffy, but becomes completely untenable once model complexity gets high and we use approaches that do not do well at explaining their decision: it basically becomes #1. A major problem with #1 is it becomes self-perpetuating and reinforcing: if everyone is going to be biased against you, you're going to do worse, and hence the models are validated/more models are created with these assumptions. Everyone using #1 may improve their own expected value at the expense of a worse outcome for society as a whole.
- tomp 6y ago
  > A major problem with #1 is it becomes self-perpetuating and reinforcing: if everyone is going to be biased against you, you're going to do worse, and hence the models are validated/more models are created with these assumptions. Everyone using #1 may improve their own expected value at the expense of a worse outcome for society as a whole. Do you have any proof of that? The world is improving on all levels, even the income gap between blacks and whites has been narrowing [1]. We're not talking about 100% segregated populations like during slavery or patriarchy, the means between the groups are very close and there's a huge overlap between the distributions. It's obvious that the vast majority of outcome isn't determined by race (otherwise you couldn't have white homeless addicts and black presidents) or any such "group" characteristic but instead by "individual" characteristics (skill, intelligence, drive, effort, hereditary wealth, chance, ...), so the evidence for this "self-perpetuating" unfairness is very weak. [1] https://www.pewsocialtrends.org/2018/07/12/income-inequality-in-the-u-s-is-rising-most-rapidly-among-asians/psdt-07-12_economic_inequality-00-06/ https://www.pewsocialtrends.org/2018/07/12/income-inequality... but you'll have to manually calculate the percentages
  - mlyle 6y ago
    > [1] https://www.pewsocialtrends.org/2018/07/12/income-inequality.. https://www.pewsocialtrends.org/2018/07/12/income-inequality.... but you'll have to manually calculate the percentages I'm completely overwhelmed by your evidence of median black income moving from 59% of whites in 1970 to 65% in 2016, and appearing to not change substantively in the last 15 years of the dataset. If the overall trend continues, we'll reach parity in another 269 years. And it hardly has been happening in a world where considering race and proxies of race is considered "OK". This is an absolutely comical piece of evidence you advance to try and justify your view.
- erik_seaberg 6y ago
  Pricing risk accurately is the lenders' job and we certainly don't want them to stop doing it. If society as a whole wants these loans to happen, society as a whole should underwrite these loans or subsidize the interest.

by tomp 6y ago

mlyle 6y ago

No, indeed, I think that's the whole point. Approach #1 is to just go to maximize expected value, using all variables, including race. Approach #2 is to use race to adjust the distributions as you suggest. Call this "affirmative action." The inbetween approach #1.5, that we usually follow, is to make a value judgment about whether any individual item is OK to use. Income? Of course it's reasonable to use income to determine whether to lend-- even if it's correlated with race. Race? No, that seems wrong. Location? Sure--- wait, you've drawn a box around all the black neighborhoods, nope! Buys certain products? Sure-- wait, you don't mean "buys products that blacks stereotypically like," do you? #1.5 is already iffy, but becomes completely untenable once model complexity gets high and we use approaches that do not do well at explaining their decision: it basically becomes #1. A major problem with #1 is it becomes self-perpetuating and reinforcing: if everyone is going to be biased against you, you're going to do worse, and hence the models are validated/more models are created with these assumptions. Everyone using #1 may improve their own expected value at the expense of a worse outcome for society as a whole.

tomp 6y ago
> A major problem with #1 is it becomes self-perpetuating and reinforcing: if everyone is going to be biased against you, you're going to do worse, and hence the models are validated/more models are created with these assumptions. Everyone using #1 may improve their own expected value at the expense of a worse outcome for society as a whole. Do you have any proof of that? The world is improving on all levels, even the income gap between blacks and whites has been narrowing [1]. We're not talking about 100% segregated populations like during slavery or patriarchy, the means between the groups are very close and there's a huge overlap between the distributions. It's obvious that the vast majority of outcome isn't determined by race (otherwise you couldn't have white homeless addicts and black presidents) or any such "group" characteristic but instead by "individual" characteristics (skill, intelligence, drive, effort, hereditary wealth, chance, ...), so the evidence for this "self-perpetuating" unfairness is very weak. [1] https://www.pewsocialtrends.org/2018/07/12/income-inequality-in-the-u-s-is-rising-most-rapidly-among-asians/psdt-07-12_economic_inequality-00-06/ https://www.pewsocialtrends.org/2018/07/12/income-inequality... but you'll have to manually calculate the percentages
- mlyle 6y ago
  > [1] https://www.pewsocialtrends.org/2018/07/12/income-inequality.. https://www.pewsocialtrends.org/2018/07/12/income-inequality.... but you'll have to manually calculate the percentages I'm completely overwhelmed by your evidence of median black income moving from 59% of whites in 1970 to 65% in 2016, and appearing to not change substantively in the last 15 years of the dataset. If the overall trend continues, we'll reach parity in another 269 years. And it hardly has been happening in a world where considering race and proxies of race is considered "OK". This is an absolutely comical piece of evidence you advance to try and justify your view.
erik_seaberg 6y ago
Pricing risk accurately is the lenders' job and we certainly don't want them to stop doing it. If society as a whole wants these loans to happen, society as a whole should underwrite these loans or subsidize the interest.