5 ms·

I think what the research itself (as opposed to politics surrounding it) was about is summed up in the last paragraph: A key result in the paper by Hardt, Pric

by qb45 10y ago

I think what the research itself (as opposed to politics surrounding it) was about is summed up in the last paragraph: A key result in the paper by Hardt, Price, and Srebro shows that—given essentially any scoring system—it's possible to efficiently find thresholds that meet any of these criteria. The crucial thing is that this "credit score" doesn't actually predict defaults quite well, because blues who are 50% likely to default have higher score than oranges who are 50% likely to default. If you base decision on a simple credit score cutoff, the blue defaulters will screw you up because of their higher score. You can defend from this by tweaking cutoffs per-group to achieve profit maximization or various "fairness" goals. In a sense, it's an attempt at producing non-garbage out from garbage in. And of course whether real-world implementations of this idea are "good" or "bad" is another can of worms altogether and varies case by case. They probably could have came up with some better examples than credit scores and labeling people by colors.

yummyfajitas 10y ago
Let me repeat what I said: Go play with the simulation to see. The various fairness criteria all achieve lower than maximal profits. The best predictor is one which is explicitly discriminatory based on race: it takes both FICO score and race into account. There are worse predictors which also discriminate explicitly based on race, but in some "fair" way. Finally, the worst predictor throws away directly relevant racial information. This is not about garbage input data - that's not the problem being addressed here at all. The input data is perfectly fine. The problem is just that the input data says "race discrimination is the best way to make money".
- qb45 10y ago
  I think you are being deceived by your real-world presumptions about the fictional example somewhat unfortunately and misleadingly engineered by Google. From your previous post: > The problem being worked on here is "what if Armenians shouldn't get car loans because they don't pay them back as much as other groups?" I.e., algorithms rightly classifying people leads to results that we believe are "unfair". No. If you stop playing with simulation and look at the input data, Armenians don't pay back less. They pay exactly as much as Iranians except that Iranians consistently have higher FICO scores because Iran infiltrated FICO with their suckxnet(TM) worm which replaces R binaries with hacked versions. Or something like that :) The general abstract idea is: you have some input "score" which is know to inaccurately predict the outcome, use the input score and measurements of its biases to produce more accurate prediction than naive threshold classifier would. And yes, the other guys talking about women's healthcare costing more or men causing more traffic accidents got it wrong too. I somewhat arbitrarily responded to you because you said something about "the real computer science problem being worked on here" and then continued to talk about other things like everybody else.
  - yummyfajitas 10y ago
    The general abstract idea is: you have some input "score" which is know to inaccurately predict the outcome, use the input score and measurements of its biases to produce more accurate prediction than naive threshold classifier would. Why don't you quote the place in the paper where they make accuracy go up, fix overfitting, or build an improved risk score? Or even just quote a place in the paper where the risk score is treated as anything other than an accurate black box? They pay exactly as much as Iranians except that Iranians consistently have higher FICO scores because Iran infiltrated FICO with their suckxnet(TM) worm which replaces R binaries with hacked versions. In the example provided in the paper (see Fig 7) that's explicitly NOT true. Blacks pay back their loans a lot less than asians/whites holding FICO fixed. For example, at a FICO score of 500, blacks pay back their loans 10% of the time while Asians do about 40%. I.e., blacks have consistently lower FICO scores because they don't pay back their loans. Further, FICO score is biased in favor of blacks. If we made it more accurate we'd be actively discriminating against blacks. For example, a black person with a financial situation reflecting a FICO of 500 would have their FICO score lowered to approx 450 to reflect their higher default rate. Did you even read the paper, or the linked article?
    - qb45 10y ago
      OK, I see what's the problem. By "higher score" I meant "higher score than the other group for given default risk", not "higher score than the other group on average". So, by my nomenclature, fig. 7.1 shows Blacks having "higher" (call it inflated) FICO scores because for every N an N% risk Black scores higher than an N% risk Asian. At the same time fig. 7.2 shows Blacks having lower scores in general, which must be because they default more, as you observed. I think we can agree that using the same cutoff for both races gives less accuracy than using higher cutoff for Blacks and lower for Asians, for some values of "higher" and "lower". You are right that "race discrimination is the best way to make money", but at the same time I think I'm right that this happens because FICO score already is racist to begin with, which is the "garbage in" I talked about.
      - yummyfajitas 10y ago

by qb45 10y ago

yummyfajitas 10y ago

Let me repeat what I said: Go play with the simulation to see. The various fairness criteria all achieve lower than maximal profits. The best predictor is one which is explicitly discriminatory based on race: it takes both FICO score and race into account. There are worse predictors which also discriminate explicitly based on race, but in some "fair" way. Finally, the worst predictor throws away directly relevant racial information. This is not about garbage input data - that's not the problem being addressed here at all. The input data is perfectly fine. The problem is just that the input data says "race discrimination is the best way to make money".

qb45 10y ago
I think you are being deceived by your real-world presumptions about the fictional example somewhat unfortunately and misleadingly engineered by Google. From your previous post: > The problem being worked on here is "what if Armenians shouldn't get car loans because they don't pay them back as much as other groups?" I.e., algorithms rightly classifying people leads to results that we believe are "unfair". No. If you stop playing with simulation and look at the input data, Armenians don't pay back less. They pay exactly as much as Iranians except that Iranians consistently have higher FICO scores because Iran infiltrated FICO with their suckxnet(TM) worm which replaces R binaries with hacked versions. Or something like that :) The general abstract idea is: you have some input "score" which is know to inaccurately predict the outcome, use the input score and measurements of its biases to produce more accurate prediction than naive threshold classifier would. And yes, the other guys talking about women's healthcare costing more or men causing more traffic accidents got it wrong too. I somewhat arbitrarily responded to you because you said something about "the real computer science problem being worked on here" and then continued to talk about other things like everybody else.
- yummyfajitas 10y ago
  The general abstract idea is: you have some input "score" which is know to inaccurately predict the outcome, use the input score and measurements of its biases to produce more accurate prediction than naive threshold classifier would. Why don't you quote the place in the paper where they make accuracy go up, fix overfitting, or build an improved risk score? Or even just quote a place in the paper where the risk score is treated as anything other than an accurate black box? They pay exactly as much as Iranians except that Iranians consistently have higher FICO scores because Iran infiltrated FICO with their suckxnet(TM) worm which replaces R binaries with hacked versions. In the example provided in the paper (see Fig 7) that's explicitly NOT true. Blacks pay back their loans a lot less than asians/whites holding FICO fixed. For example, at a FICO score of 500, blacks pay back their loans 10% of the time while Asians do about 40%. I.e., blacks have consistently lower FICO scores because they don't pay back their loans. Further, FICO score is biased in favor of blacks. If we made it more accurate we'd be actively discriminating against blacks. For example, a black person with a financial situation reflecting a FICO of 500 would have their FICO score lowered to approx 450 to reflect their higher default rate. Did you even read the paper, or the linked article?
  - qb45 10y ago
    OK, I see what's the problem. By "higher score" I meant "higher score than the other group for given default risk", not "higher score than the other group on average". So, by my nomenclature, fig. 7.1 shows Blacks having "higher" (call it inflated) FICO scores because for every N an N% risk Black scores higher than an N% risk Asian. At the same time fig. 7.2 shows Blacks having lower scores in general, which must be because they default more, as you observed. I think we can agree that using the same cutoff for both races gives less accuracy than using higher cutoff for Blacks and lower for Asians, for some values of "higher" and "lower". You are right that "race discrimination is the best way to make money", but at the same time I think I'm right that this happens because FICO score already is racist to begin with, which is the "garbage in" I talked about.
    - yummyfajitas 10y ago