4 ms·

I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. Sorry, s/loa

I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. Sorry, s/loan/allow out on bail/ or s/loan/show advertisement/ to go with examples in the article. Mathematically it's the same problem. I just default to thinking about loans since I've worked on loans and since most of the academic work on the problem is about lending. But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality... Thanks - you've perfectly illustrated my point as to why the term "bias" is misleading. The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case.

smallnamespace 10y ago
> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case. You can make a strong argument that, for example, the FB news feed algorithm leads to results that are unrepresentative of reality, because its input data (user clicks and likes) do not sufficiently represent reality accurately. Or from the article: "Nikon’s confusion about Asian faces and HP’s skin tone issues in their face recognition software both seem to be the product of learning from skewed example sets." Nowhere here is the algorithm being blamed, but we have an example of results that at least some people find objectionable due to patterns in the input data. Just to take a particular example, the algorithm's output has marked someone as blinking when they aren't, they're just Asian. That's a particular example of results being unrepresentative of reality. As much as you want to talk about the algorithms in isolation, using ML also includes figuring out what input data to use and also how to properly interpret the output.
notahacker 10y ago
> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case. But unlike the loan example they are; that's precisely my point. In the loan example the algorithms (or actuaries' calculations) are often entirely accurate in identifying that for whatever reason $ISBLACK is associated with statistically significant increased likelihood of default after controlling for other variables, which may result in an unbiased estimation model assessing a particular person falls into an unacceptable default risk category (which FWIW isn't the same thing as accurately predicting a particular person won't pay, which is a more dangerous conflation of concepts...) That doesn't apply to the examples discussed in the article - where Asians aren't blinking (or statistically more likely to be blinking), people trying to game a Twitter bot's corpus of conversations are not representative of human interaction patterns, nobody seriously believes social media bubbles represent reality and even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white.
- AnthonyMouse 10y ago
  > even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white. That's actually the expected consequence of having different recidivism rates. If some of the predictions of who will recidivate are wrong at random and a higher percentage of black people were predicted to recidivate then a higher percentage of black people will have been incorrectly predicted to recidivate. If some of the predictions of who won't recidivate are wrong at random and a higher percentage of white people were predicted not to recidivate then a higher percentage of white people will have been incorrectly predicted not to recidivate. In other words, X * error rate is more than Y * error rate because X is more than Y.

smallnamespace 10y ago

> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case. You can make a strong argument that, for example, the FB news feed algorithm leads to results that are unrepresentative of reality, because its input data (user clicks and likes) do not sufficiently represent reality accurately. Or from the article: "Nikon’s confusion about Asian faces and HP’s skin tone issues in their face recognition software both seem to be the product of learning from skewed example sets." Nowhere here is the algorithm being blamed, but we have an example of results that at least some people find objectionable due to patterns in the input data. Just to take a particular example, the algorithm's output has marked someone as blinking when they aren't, they're just Asian. That's a particular example of results being unrepresentative of reality. As much as you want to talk about the algorithms in isolation, using ML also includes figuring out what input data to use and also how to properly interpret the output.

notahacker 10y ago

> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case. But unlike the loan example they are; that's precisely my point. In the loan example the algorithms (or actuaries' calculations) are often entirely accurate in identifying that for whatever reason $ISBLACK is associated with statistically significant increased likelihood of default after controlling for other variables, which may result in an unbiased estimation model assessing a particular person falls into an unacceptable default risk category (which FWIW isn't the same thing as accurately predicting a particular person won't pay, which is a more dangerous conflation of concepts...) That doesn't apply to the examples discussed in the article - where Asians aren't blinking (or statistically more likely to be blinking), people trying to game a Twitter bot's corpus of conversations are not representative of human interaction patterns, nobody seriously believes social media bubbles represent reality and even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white.

AnthonyMouse 10y ago
> even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white. That's actually the expected consequence of having different recidivism rates. If some of the predictions of who will recidivate are wrong at random and a higher percentage of black people were predicted to recidivate then a higher percentage of black people will have been incorrectly predicted to recidivate. If some of the predictions of who won't recidivate are wrong at random and a higher percentage of white people were predicted not to recidivate then a higher percentage of white people will have been incorrectly predicted not to recidivate. In other words, X * error rate is more than Y * error rate because X is more than Y.