4 ms·
I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. But what "sep
by notahacker 10y ago
I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses?
- AnthonyMouse 10y ago> I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. It's an example. The article mentions calculating recidivism rates to determine parole, which is a similar problem if you prefer that one. > But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses? Most of the examples in the article have nothing to do with "bias" and are really the principal-agent problem. Misalignment of incentives. Some website wants to maximize clicks rather than act in the user's interest, they create a machine to maximize clicks, the machine maximizes clicks. It's doing what they wanted it to do -- it's just not doing what you wanted it to do. This category of problem has almost nothing to do with ML in particular. It's like identifying that charlatans often wear cotton and writing an article on the link between cotton and charlatans.
- smallnamespace 10y ago> This category of problem has almost nothing to do with ML in particular. The article is identifying a particular misconception that people might have, which is because a computer did it, the result will be (racially) unbiased. It might be obvious to you that's not the case, since you're familiar with the ML techniques in question, but still valuable to a lay audience to make that point crystal clear.
- notahacker 10y agoNews algorithms tending to classifies stories as relevant reading based on relatively crude analysis of word frequencies and headline clickthroughs (as opposed to, say, whether the story added additional information or an alternative perspective, both of which would typically be negatively weighted by such an algorithm) is much more an ML problem than a principal-agent problem, unless one believes that "maximising clicks" is literally all anyone applying this technique aims to do, and that's the only reason why ML systems tend to optimise for simple goals like click-maximization rather than complex ones like breadth and salience of relevant information
- AnthonyMouse 10y agoThere are obviously still unsolved problems in ML. But Facebook has billions of dollars. It isn't that they couldn't solve the problem if they wanted to. Yet they haven't.
- yummyfajitas 10y agoI'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. Sorry, s/loan/allow out on bail/ or s/loan/show advertisement/ to go with examples in the article. Mathematically it's the same problem. I just default to thinking about loans since I've worked on loans and since most of the academic work on the problem is about lending. But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality... Thanks - you've perfectly illustrated my point as to why the term "bias" is misleading. The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case.
- smallnamespace 10y ago> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case. You can make a strong argument that, for example, the FB news feed algorithm leads to results that are unrepresentative of reality, because its input data (user clicks and likes) do not sufficiently represent reality accurately. Or from the article: "Nikon’s confusion about Asian faces and HP’s skin tone issues in their face recognition software both seem to be the product of learning from skewed example sets." Nowhere here is the algorithm being blamed, but we have an example of results that at least some people find objectionable due to patterns in the input data. Just to take a particular example, the algorithm's output has marked someone as blinking when they aren't, they're just Asian. That's a particular example of results being unrepresentative of reality. As much as you want to talk about the algorithms in isolation, using ML also includes figuring out what input data to use and also how to properly interpret the output.
- notahacker 10y ago> The term "bias" leads you to believe the algorithms are generating results unrepresentative of wider reality, which simply isn't the case. But unlike the loan example they are; that's precisely my point. In the loan example the algorithms (or actuaries' calculations) are often entirely accurate in identifying that for whatever reason $ISBLACK is associated with statistically significant increased likelihood of default after controlling for other variables, which may result in an unbiased estimation model assessing a particular person falls into an unacceptable default risk category (which FWIW isn't the same thing as accurately predicting a particular person won't pay, which is a more dangerous conflation of concepts...) That doesn't apply to the examples discussed in the article - where Asians aren't blinking (or statistically more likely to be blinking), people trying to game a Twitter bot's corpus of conversations are not representative of human interaction patterns, nobody seriously believes social media bubbles represent reality and even the very important recidivism risk model is 50% more likely to be a false positive if a suspected repeat offender is black, and less likely to be a false negative if the individual categorised as low risk is white.