6 ms·

The paper's authors have the best of intentions, but to me race-aware algorithms programed for social justice sounds completely dystopian. If a black man and a

The paper's authors have the best of intentions, but to me race-aware algorithms programed for social justice sounds completely dystopian. If a black man and a white man rob a bank together, is it fair that they receive the same sentence? Or is it fair that a sentencing algorithm give the black man probation and send the white man to jail to correct for disparate outcomes based on race?

cyberbanjo 8y ago
Ironically I think algorithms would do the opposite, with the training data that is current justice system antics.
raverbashing 8y ago
It's not so much about programming for social justice, but accounting for biases that are present in the current status quo and that end up in training data There was a recent talk from a Meetup engineer about this but I can't find the link right now.
- danieltillett 8y ago
  Then shouldn't the algorithms incorporate the unbiased data rather than incorporate something we know is biased to correct for other biases? A question. How do we know the algorithm is biased if it give a racial disparate result? Don't tell me the evidence is that the result is racial disparate.
  - raverbashing 8y ago
    There usually isn't unbiased data (think: law enforcement databases), though you could maybe remove the variables more likely to be discriminated against and train on that There are statistical analysis that can answer your second question, and it's not only based on disparate results (though that's probably part of it). And the solution is not "changing the signal" of the bias
    - danieltillett 8y ago
      If there are statistical analysis that can answer if the data is biased why isn't the focus of attention all biases rather than just biases that may result in racial discrimination? Shouldn't all the arguments be over the data and any biases present (and how to obtain unbiased data) and not the result?
woodruffw 8y ago
Affirmative action stands for affirmatively elevating the status of disenfranchised groups to equality, not above it. In the example you gave, that would mean giving both of them probation, since probation is what the privileged group is afforded. (As an aside: our current eagerness to imprison people for relatively minor crimes strikes me as fundamentally unfair, especially in light of our inability to provide corrective justice. As such, while it would be equal to send both to prison for robbery, I don't think it would be correct to call it fair.)
- lambdadmitry 8y ago
  That doesn't really work. It's impossible to restore individual justice by using statistical numbers as KPI. How'd you know the system became "fair"? By comparing the racial ratio of incarceration to the ratio in the general population? That doesn't make any sense if you account for disproportionate representation of black people on the lower end of income scale, which is generally associated with violent crime and reoffending. What target level of disproportionate incarceration of blacks would you target? Or would you prefer giving white people harsher sentences just for the colour of their skin? Statistics may point to problems (so we do investigate why exactly black people are more likely to get in prison), but simple ratios aren't nearly enough to bring justice.
  - woodruffw 8y ago
    I'm not interested in ratios or statistics when it comes to justice. I'm interested in fairness, which entails doing right by individuals. One of the ways that we can do right by individuals, especially individuals who have been and are currently disenfranchised, is to take affirmative steps towards improving their lives. One of those affirmative steps (and a relatively easy one at that!) is simply not punishing them more harshly than we would the privileged. Nothing about this entails treating white people (or any particular privileged group) worse.
obastani 8y ago
The problem you mentioned is not the one that is being tackled. The real challenge is that being blind to just race isn't enough. For example, zip code is often highly correlated with race, so using zip code to predict whether an applicant would repay a loan may implicitly be discriminatory. We need to know race to determine whether a given covariate is okay to use. But we would usually not use race to predict outcomes (except possibly in a limited number of affirmative action settings, such as college admissions).
- whatshisface 8y ago
  Ok, help me reason though this. Let's say that native americans are poorer on average and therefore less likely to repay a $1k loan. Therefore, if you give out loans based on the probability that someone will be able to pay them back you will be discriminating against native americans (not the individual native americans with good credit scores, but you will be denying more native americans on average). So, you shouldn't try to compute how likely it is for someone to pay their loan back. I keep hearing things like, "you can't use zipcodes as input because zipcodes are correlated with race and it is illegal to discriminate on race." However, some races have better financial situations on average, which means that financial situations correlate with race, and by the same argument you shouldn't take finances into account when you review a loan application. Obviously that is absurd, but where does it go wrong?
  - rootusrootus 8y ago
    Your personal financial history is a lot more specific than your zip code. They may be on different ends of the same scale, but it seems far more defensible to make choices based on financial situation than it does zip code.
    - whatshisface 8y ago
      Setting aside for a moment the fact that your financial history includes your home loan which includes your home as collateral which is located in your zipcode, suppose that the model got more accuracy by looking at the zipcode. Isn't that all the defense that's needed? What is meant, exactly, by "defense?"

cyberbanjo 8y ago

Ironically I think algorithms would do the opposite, with the training data that is current justice system antics.

raverbashing 8y ago

It's not so much about programming for social justice, but accounting for biases that are present in the current status quo and that end up in training data There was a recent talk from a Meetup engineer about this but I can't find the link right now.

danieltillett 8y ago
Then shouldn't the algorithms incorporate the unbiased data rather than incorporate something we know is biased to correct for other biases? A question. How do we know the algorithm is biased if it give a racial disparate result? Don't tell me the evidence is that the result is racial disparate.
- raverbashing 8y ago
  There usually isn't unbiased data (think: law enforcement databases), though you could maybe remove the variables more likely to be discriminated against and train on that There are statistical analysis that can answer your second question, and it's not only based on disparate results (though that's probably part of it). And the solution is not "changing the signal" of the bias
  - danieltillett 8y ago
    If there are statistical analysis that can answer if the data is biased why isn't the focus of attention all biases rather than just biases that may result in racial discrimination? Shouldn't all the arguments be over the data and any biases present (and how to obtain unbiased data) and not the result?

woodruffw 8y ago

Affirmative action stands for affirmatively elevating the status of disenfranchised groups to equality, not above it. In the example you gave, that would mean giving both of them probation, since probation is what the privileged group is afforded. (As an aside: our current eagerness to imprison people for relatively minor crimes strikes me as fundamentally unfair, especially in light of our inability to provide corrective justice. As such, while it would be equal to send both to prison for robbery, I don't think it would be correct to call it fair.)

lambdadmitry 8y ago
That doesn't really work. It's impossible to restore individual justice by using statistical numbers as KPI. How'd you know the system became "fair"? By comparing the racial ratio of incarceration to the ratio in the general population? That doesn't make any sense if you account for disproportionate representation of black people on the lower end of income scale, which is generally associated with violent crime and reoffending. What target level of disproportionate incarceration of blacks would you target? Or would you prefer giving white people harsher sentences just for the colour of their skin? Statistics may point to problems (so we do investigate why exactly black people are more likely to get in prison), but simple ratios aren't nearly enough to bring justice.
- woodruffw 8y ago
  I'm not interested in ratios or statistics when it comes to justice. I'm interested in fairness, which entails doing right by individuals. One of the ways that we can do right by individuals, especially individuals who have been and are currently disenfranchised, is to take affirmative steps towards improving their lives. One of those affirmative steps (and a relatively easy one at that!) is simply not punishing them more harshly than we would the privileged. Nothing about this entails treating white people (or any particular privileged group) worse.

obastani 8y ago

The problem you mentioned is not the one that is being tackled. The real challenge is that being blind to just race isn't enough. For example, zip code is often highly correlated with race, so using zip code to predict whether an applicant would repay a loan may implicitly be discriminatory. We need to know race to determine whether a given covariate is okay to use. But we would usually not use race to predict outcomes (except possibly in a limited number of affirmative action settings, such as college admissions).

whatshisface 8y ago
Ok, help me reason though this. Let's say that native americans are poorer on average and therefore less likely to repay a $1k loan. Therefore, if you give out loans based on the probability that someone will be able to pay them back you will be discriminating against native americans (not the individual native americans with good credit scores, but you will be denying more native americans on average). So, you shouldn't try to compute how likely it is for someone to pay their loan back. I keep hearing things like, "you can't use zipcodes as input because zipcodes are correlated with race and it is illegal to discriminate on race." However, some races have better financial situations on average, which means that financial situations correlate with race, and by the same argument you shouldn't take finances into account when you review a loan application. Obviously that is absurd, but where does it go wrong?
- rootusrootus 8y ago
  Your personal financial history is a lot more specific than your zip code. They may be on different ends of the same scale, but it seems far more defensible to make choices based on financial situation than it does zip code.
  - whatshisface 8y ago
    Setting aside for a moment the fact that your financial history includes your home loan which includes your home as collateral which is located in your zipcode, suppose that the model got more accuracy by looking at the zipcode. Isn't that all the defense that's needed? What is meant, exactly, by "defense?"