4 ms·

> That's not the point -- the problem is that conflating technical and non-technical definitions is a way to avoid having that discussion. On this we can 100%

> That's not the point -- the problem is that conflating technical and non-technical definitions is a way to avoid having that discussion. On this we can 100% agree. Choosing "bias" to mean "statistical bias", and then actively ignoring anything else that is currently called bias, is an extremely political choice. Are you seriously asserting that naming clashes validate political opinions, or that some large group of people's political opinions would suddenly change if only they knew the statistical meaning of bias?

yummyfajitas 10y ago
No one said we should ignore this other concept. All I said is that we should name it something else. Are you seriously asserting that naming clashes validate political opinions, or that some large group of people's political opinions would suddenly change if only they knew the statistical meaning of bias? Yes, I think this use of "bias" is meant to mislead. Specifically, I think the term "bias" is used to make people think machines are drawing incorrect conclusions (that's what "bias" means in the dictionary and in most common uses of the term). There are two separate concepts: 1) An algorithm refuses to issue loans to blacks even though those blacks would pay the loans back. 2) An algorithm refuses to issue loans to blacks because they would not pay the loans back. You want to use the term "bias" to describe both situations; I think this will mislead people. Specifically most people will think "bias" means we live in situation (1). I want to use "bias" to refer to (1) and to use a separate term to refer to (2). Why do you want to conflate these two situations?
- smallnamespace 10y ago
  Replace the word 'algorithm' with 'person' or 'institution'. The politics around racial discrimination and bias means that in reality, few people do the work to tease apart (1) and (2). In particular, I think many people are extremely uncomfortable with the consequences that (2) being true entail. Hence 'racial bias' usually has a simpler meaning: something is biased if the decision conditioned on race is not identical to the unconditional decision. That encompasses both (1) and (2). You seem to ascribe some sort of malicious motive to the authors, when what you really have a problem with is how the country talks about race and racial bias in general. The point is that in general, statistically unbiased algorithms will not lead to racially unbiased (in the strict conditional outcome = unconditional outcome sense) results. You might think this is completely obvious as a practitioner in the field, but it is far from obvious to the general public. Hence why it's fair to point that out to the public.
- throwaway729 10y ago
  (edit: Either confusion isn't a real problem, or else the confusion is inevitable and some new term will become the politically charged one with lots of contested meanings. In either case, the correct solution from your perspective should be more statistics education for more people rather than trying to change terminology.) > All I said is that we should name it something else As I said elsewhere, some mathematician would come up with a concept, name it after that new thing, and then we're back here all over again because "bias" is inherently moral and political and has shades of meaning informed by moral and political opinion. You won't have solved the precision problem, or even the overloaded terms problem. > Specifically, I think the term "bias" is used to make people think machines are drawing incorrect conclusions.. I think this will mislead people I agree "bias" is a politically powerful word. I don't agree, however, that people fail to know and understand the difference between (1) and (2). And I also think all people know that the term "bias" could mean both, and manage to communicate with one another using the full range of their vocabulary whenever it's unclear what's being discussed. See e.g. internet discussions around stop and frisk or WoD or racial profiling at airports. People understand the difference between (1) and (2), even when bias is used to refer to both. I might be wrong about that. A far more important prediction: I don't think there's anyone -- on either side of the debate -- whose position can be explained by their misunderstanding of the statistical meaning of "bias". People understand the statistical meaning (or not), and call (2) bias regardless. Because they aren't referring to a mathematical object, they're referring to the effect something has within a larger social and political context. A context that cannot be perfectly modeled, and so is necessarily vague and difficult to pin down. The fact that "bias" can have both meanings isn't a real problem. People are good communicators and, when they want, can easily distinguish between (1) and (2). Saying that (2) is "not bias" is definitely a political statement when taking the common definition of bias. People are free to agree or disagree with that statement, but arguing via way of naming clash isn't a compelling argument (not saying you're making that argument, but GP is I think). > Why do you want to conflate these two situations? I don't think this is a real problem.
- notahacker 10y ago
  I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses?
  - AnthonyMouse 10y ago
    > I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. It's an example. The article mentions calculating recidivism rates to determine parole, which is a similar problem if you prefer that one. > But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses? Most of the examples in the article have nothing to do with "bias" and are really the principal-agent problem. Misalignment of incentives. Some website wants to maximize clicks rather than act in the user's interest, they create a machine to maximize clicks, the machine maximizes clicks. It's doing what they wanted it to do -- it's just not doing what you wanted it to do. This category of problem has almost nothing to do with ML in particular. It's like identifying that charlatans often wear cotton and writing an article on the link between cotton and charlatans.
    - smallnamespace 10y ago
      > This category of problem has almost nothing to do with ML in particular. The article is identifying a particular misconception that people might have, which is because a computer did it, the result will be (racially) unbiased. It might be obvious to you that's not the case, since you're familiar with the ML techniques in question, but still valuable to a lay audience to make that point crystal clear.
    - notahacker 10y ago
      News algorithms tending to classifies stories as relevant reading based on relatively crude analysis of word frequencies and headline clickthroughs (as opposed to, say, whether the story added additional information or an alternative perspective, both of which would typically be negatively weighted by such an algorithm) is much more an ML problem than a principal-agent problem, unless one believes that "maximising clicks" is literally all anyone applying this technique aims to do, and that's the only reason why ML systems tend to optimise for simple goals like click-maximization rather than complex ones like breadth and salience of relevant information

yummyfajitas 10y ago

No one said we should ignore this other concept. All I said is that we should name it something else. Are you seriously asserting that naming clashes validate political opinions, or that some large group of people's political opinions would suddenly change if only they knew the statistical meaning of bias? Yes, I think this use of "bias" is meant to mislead. Specifically, I think the term "bias" is used to make people think machines are drawing incorrect conclusions (that's what "bias" means in the dictionary and in most common uses of the term). There are two separate concepts: 1) An algorithm refuses to issue loans to blacks even though those blacks would pay the loans back. 2) An algorithm refuses to issue loans to blacks because they would not pay the loans back. You want to use the term "bias" to describe both situations; I think this will mislead people. Specifically most people will think "bias" means we live in situation (1). I want to use "bias" to refer to (1) and to use a separate term to refer to (2). Why do you want to conflate these two situations?

smallnamespace 10y ago
Replace the word 'algorithm' with 'person' or 'institution'. The politics around racial discrimination and bias means that in reality, few people do the work to tease apart (1) and (2). In particular, I think many people are extremely uncomfortable with the consequences that (2) being true entail. Hence 'racial bias' usually has a simpler meaning: something is biased if the decision conditioned on race is not identical to the unconditional decision. That encompasses both (1) and (2). You seem to ascribe some sort of malicious motive to the authors, when what you really have a problem with is how the country talks about race and racial bias in general. The point is that in general, statistically unbiased algorithms will not lead to racially unbiased (in the strict conditional outcome = unconditional outcome sense) results. You might think this is completely obvious as a practitioner in the field, but it is far from obvious to the general public. Hence why it's fair to point that out to the public.
throwaway729 10y ago
(edit: Either confusion isn't a real problem, or else the confusion is inevitable and some new term will become the politically charged one with lots of contested meanings. In either case, the correct solution from your perspective should be more statistics education for more people rather than trying to change terminology.) > All I said is that we should name it something else As I said elsewhere, some mathematician would come up with a concept, name it after that new thing, and then we're back here all over again because "bias" is inherently moral and political and has shades of meaning informed by moral and political opinion. You won't have solved the precision problem, or even the overloaded terms problem. > Specifically, I think the term "bias" is used to make people think machines are drawing incorrect conclusions.. I think this will mislead people I agree "bias" is a politically powerful word. I don't agree, however, that people fail to know and understand the difference between (1) and (2). And I also think all people know that the term "bias" could mean both, and manage to communicate with one another using the full range of their vocabulary whenever it's unclear what's being discussed. See e.g. internet discussions around stop and frisk or WoD or racial profiling at airports. People understand the difference between (1) and (2), even when bias is used to refer to both. I might be wrong about that. A far more important prediction: I don't think there's anyone -- on either side of the debate -- whose position can be explained by their misunderstanding of the statistical meaning of "bias". People understand the statistical meaning (or not), and call (2) bias regardless. Because they aren't referring to a mathematical object, they're referring to the effect something has within a larger social and political context. A context that cannot be perfectly modeled, and so is necessarily vague and difficult to pin down. The fact that "bias" can have both meanings isn't a real problem. People are good communicators and, when they want, can easily distinguish between (1) and (2). Saying that (2) is "not bias" is definitely a political statement when taking the common definition of bias. People are free to agree or disagree with that statement, but arguing via way of naming clash isn't a compelling argument (not saying you're making that argument, but GP is I think). > Why do you want to conflate these two situations? I don't think this is a real problem.
notahacker 10y ago
I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses?
- AnthonyMouse 10y ago
  > I'm perplexed by your examples (1) and (2) since the article doesn't actually discuss loaning to defaulting black people in any way shape or form. It's an example. The article mentions calculating recidivism rates to determine parole, which is a similar problem if you prefer that one. > But what "separate term" would you consider to be more appropriate to describe ML systems' tendency to generate results unrepresentative of wider reality as a result of the characteristics of the subset it's trained on or inadequacies in its specification, which is what the article actually discusses? Most of the examples in the article have nothing to do with "bias" and are really the principal-agent problem. Misalignment of incentives. Some website wants to maximize clicks rather than act in the user's interest, they create a machine to maximize clicks, the machine maximizes clicks. It's doing what they wanted it to do -- it's just not doing what you wanted it to do. This category of problem has almost nothing to do with ML in particular. It's like identifying that charlatans often wear cotton and writing an article on the link between cotton and charlatans.
  - smallnamespace 10y ago
    > This category of problem has almost nothing to do with ML in particular. The article is identifying a particular misconception that people might have, which is because a computer did it, the result will be (racially) unbiased. It might be obvious to you that's not the case, since you're familiar with the ML techniques in question, but still valuable to a lay audience to make that point crystal clear.
  - notahacker 10y ago
    News algorithms tending to classifies stories as relevant reading based on relatively crude analysis of word frequencies and headline clickthroughs (as opposed to, say, whether the story added additional information or an alternative perspective, both of which would typically be negatively weighted by such an algorithm) is much more an ML problem than a principal-agent problem, unless one believes that "maximising clicks" is literally all anyone applying this technique aims to do, and that's the only reason why ML systems tend to optimise for simple goals like click-maximization rather than complex ones like breadth and salience of relevant information