20 ms·

I think we need to stop using the word "bias" and "prejudice". The term "bias" has a real, well-defined meaning in statistics, and both of these terms imply inc

I think we need to stop using the word "bias" and "prejudice". The term "bias" has a real, well-defined meaning in statistics, and both of these terms imply incorrect decisions are being systematically made. Only one example in this article is actually about incorrect decisions. Most examples are about unbiased learning systems doing exactly what they are designed to do, but people other than the designers wishing they did different things. For example, consider "conflicting goals bias". This isn't a bias - the system is maximizing clicks exactly as desired. It's just that some random third party wishes the system were actually trying to mitigate a nonexistent psychological problem (namely stereotype threat) instead [1]. What they call "similarity bias" is the same thing. The system is attempting to show people stories they like (and probably does a good job at it), but the author wishes instead that the selection of stories was closer to what a journalist might choose. Another social "bias" - namely "redlining"/redundant encoding/etc is actually the elimination of statistical bias. A lot of input metrics - e.g. SAT score, FICO score, etc - are biased in favor of non-Asian minorities (and against Asians) [2] and machine learning algorithms designed to find hidden patterns discover this bias and fix it. Conflating "someone is doing X but I wish they did Y" with bias is not useful. It's also not useful to conflate the elimination of socially desirable biases with introducing new biases. [1] Stereotype threat has repeatedly failed to replicate. Funnel plots suggest it only existed to begin with due to publication bias. http://www.sciencedirect.com.sci-hub.cc/science/article/pii/S0022440514000831 http://www.sciencedirect.com.sci-hub.cc/science/article/pii/... https://dl.dropboxusercontent.com/u/85192141/2013-ganley.pdf https://dl.dropboxusercontent.com/u/85192141/2013-ganley.pdf https://en.wikipedia.org/wiki/Stereotype_threat#Failures_to_replicate_and_publication_bias https://en.wikipedia.org/wiki/Stereotype_threat#Failures_to_... https://replicationindex.wordpress.com/tag/stereotype-threat-and-womens-math-performance/ https://replicationindex.wordpress.com/tag/stereotype-threat... [2] See for example Figure 7 here: https://drive.google.com/file/d/0B-wQVEjH9yuhanpyQjUwQS1JOTQ/view https://drive.google.com/file/d/0B-wQVEjH9yuhanpyQjUwQS1JOTQ...

deleted 10y ago
[deleted]
zkhalique 10y ago
Yeah, the word bias seems to be overloaded with many meanings. I actually wonder if there is any philosophical difference between the word "knowledge" and the word "bias". But about machine learning... a system can learn from a set of outcomes which are the result of a "biased" system, that is to say tainted with an incorrect Bayesian prior which is not properly corrected, such as let's say courts in a racist society or whatever. It would learn the same bias. Because its goal is to maximize compatibility with the outcomes that the humans did. So it perpetuates those weights. The problem is that we don't know whether the human decisions matched the reality. "Did the person commit the crime" for example. We might have to wait until more unbiased estimators for such activity come along, and throw away old historical data. It's sort of like when Black-Scholes became a self-fulfilling prophecy for valuing derivatives, but only after it became widespread. The market started using Black-Scholes to value derivatives, so it became the best model to predict the value of derivatives. But until then, other models might have fit the historical data better.
- yummyfajitas 10y ago
  Yeah, the word bias seems to be overloaded with many meanings. I actually wonder if there is any philosophical difference between the word "knowledge" and the word "bias". If you interpret the word "bias" statistically - rather than the way innumerate reporters trying to sound profound use it - then it's pretty straightforward to do so. "Knowledge" represents a fixed snapshot of your information about the world. "Bias" represents a tendency for your knowledge to systematically differ from reality even as more data is gathered. In programming terms, knowledge is the data in your DB at a fixe dtime. Bias is a tendency for my inserts to fail while yours succeed. But about machine learning... a system can learn from a set of outcomes which are the result of a "biased" system, that is to say tainted with an incorrect Bayesian prior which is not properly corrected, such as let's say courts in a racist society or whatever. It would learn the same bias. Because its goal is to maximize compatibility with the outcomes that the humans did. So it perpetuates those weights. If the goal of a system is to predict human behaviors, then it will in fact do that. That's not "bias", that's just building a system with the goal of matching human behavior and getting what you asked for. However, if the inputs to your algorithm are biased predictors of the output your outputs can still be unbiased. The tendency of machine learning systems is to detect hidden patterns in the data; biased inputs are just another pattern. I wrote a blog post earlier this year explaining this in detail: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_...
Moshe_Silnorin 10y ago
Given the findings on stereotype accuracy are some of the best-replicated in psychology, I've always found stereotype threat a humorous idea. It implies stereotypes are accurate because they trick us into perpetuating them. Can you think of a more absurd conspiracy theory? It's in keeping with the very popular notion that you can change reality with words, and thus is commonly said to be true in hopes of making it so.
pessimizer 10y ago
> The term "bias" has a real, well-defined meaning in statistics, and both of these terms imply incorrect decisions are being systematically made. Bias is not a word that was invented by statisticians[1], it's a word that statisticians adopted. It's just as legitimate to say that when a machine learning technique is trained with biased input that it produces biased output as it is to say that some particular machine learning method produces biased output from unbiased input. Either way, it's perfectly fine to say that the use of that technique results in bias even if that bias is in an aspect of the output that the algorithm wasn't selected to optimize. It seems that what you're arguing is that the algorithm shouldn't be blamed in these cases, but in addition, you're declaring that there's no such thing as prejudice or bias, and implying that what the algorithms are doing is pulling some (uncomfortable for gatekeepers) platonic truth out of the ether (without any acknowledgement of the possibility of biased input.) Or more simply, you're using this article as an excuse to push a favorite "race realist" agenda which you use to justify a purist libertarianism (and which I agree it requires.) [1] http://etymonline.com/index.php?term=bias http://etymonline.com/index.php?term=bias
- yummyfajitas 10y ago
  The term "bias" and "prejudice" even outside statistics are defined by the dictionary as being about getting the wrong results: "preconceived opinion that is not based on reason or actual experience". This agrees more or less with the statistical term, it's just less precise. That's also NOT what machine learning algorithms do. If you want to declare an algorithm biased because it optimizes what it was designed to optimize rather than some random tangential goal, then the term has become meaningless. Similarly, my cell phone is broken because it can only make calls and is ineffective at pulling trains. A locomotive is also broken because it doesn't make tacos. It seems that what you're arguing is that the algorithm shouldn't be blamed in these cases, but in addition, you're declaring that there's no such thing as prejudice or bias...what the algorithms are doing is pulling some (uncomfortable for gatekeepers) platonic truth out of the ether (without any acknowledgement of the possibility of biased input.) I have no idea how you could possibly think I made this claim given that I explicitly listed two examples of algorithms which are biased. I then hinted at how algorithms can eliminate this bias. Consider rereading what I wrote. If you want more details on how algorithms correct biased inputs (it's not based on "ether") read the blog post I linked to downthread: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_... Or more simply, you're using this article as an excuse to push a favorite "race realist" agenda which you use to justify a purist libertarianism (and which I agree it requires.) I have no idea why you think "race realist" agendas require purist libertarianism, or vice versa. That's a random political tangent and totally unrelated to this conversation. I'm beginning to think you are seeking to derail the conversation rather than discussing statistics in good faith. In the hopes that I've misunderstood you, I will nevertheless provide you with a good faith clarification. I made no normative claims regarding libertarianism or anything else. The only normative claim I've made is that we should not wrongfully apply an existing term (bias) to completely unrelated concepts (having undesired social outcomes).
throwaway729 10y ago
When there exist two sets of meanings for a word -- one vernacular, the other technical -- we shouldn't try to insist that the technical meaning over-ride the vernacular meaning. That's almost always a losing battle. Insisting that a system isn't biased in the vernacular sense due to a particular technical notion of bias completely misses the point. And we almost never have enough understanding about the world to make a definitive case that the system isn't biased in the vernacular sense.
- yummyfajitas 10y ago
  My claim is, however, that when discussing statistics we should stick to the statistics terms. If we want to discuss a new concept then we should make up a new word for it. To do otherwise risks having people get confused as to what underlying concepts are actually being discussed. For instance, go read the HN comments here: https://news.ycombinator.com/item?id=13004790 https://news.ycombinator.com/item?id=13004790 Why risk such confusion when we can just use two different words for two different concepts? However, you are also wrong about the vernacular sense, at least if the dictionary is to be believed: Bias: prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair. Prejudice: preconceived opinion that is not based on reason or actual experience. This are just informal versions of the statistical term. To translate this to another topic, if we are discussing global warming I don't think we should use the term "warm" to reference "feelings in my heart". Why would we want to confuse whether we are discussing the temperature of the earth or my feelings?
  - throwaway729 10y ago
    If this were a conversation about a statistics paper or textbook, or a particular ML algorithm, you might have a point. But we're not discussing statistics. We're discussing the social impact of ML algorithms. > Why risk such confusion when we can just use two different words for two different concepts? The problem with this is that we'll always have that confusion because most people aren't statistics experts. So it's up to the domain experts to know when one meaning or the other is intended. That's true in every domain. > However, you are also wrong about the vernacular sense, at least if the dictionary is to be believed Dictionary definitions don't fully capture the vernacular meaning of words -- especially politically charged words such as "bias". > This are just informal versions of the statistical term. But when real people use the term bias in real conversations, they aren't typically approximating the statistical meaning. Rather, they are typically referring to some a priori ethical limitations on decision making. For example, most people would say that taking skin color into account when approving a housing loan is a form of bias. But they don't mean that in any particularly technical way. There absolutely is moral and political content in the term "bias". For example, when equal lending laws were passed, an unbiased algorithm definitely would've considered race when choosing the outcome of a loan application. African Americans were systematically discriminated against in employment decisions, and often the first to be let go when hard times hit. So a completely unbiased loan officer could come to the statistically valid conclusion that people of color were far more likely to default. None-the-less, a vase majority of people today would agree that the loan officer was exhibiting racial bias. And they're not coming to that conclusion due to innumeracy or some vacuous personal preference. They're perceiving a vast network of causal relationships between decisions that reinforce a systematic bias, even when many of those decisions aren't locally statistically biased > I don't think we should use the term "warm" to reference "feelings in my heart". And I agree, but I agree because non-experts know and understand the difference. And because the term "global warming" with the alternative meaning isn't already in wide-spread vernacular use.
caspianm 10y ago
Say I have two categories, programming articles and non- programming articles, and some other data about each article. And I want to predict whether the article will be interesting or not. And I want to be fair to interesting non-programming articles by having the same proportion of false negatives to correct positives in the non-programming subset of articles as in the programming subset of articles. Is there a technical term for that in statistics? It's like trying to get a representative sample, but only representative in one specific way (topic), and deliberately non representative in another (interestingness) I think this could get at one of the things people mean, and it might be interesting to see how this trades off against overall accuracy or representativeness in other categories.

deleted 10y ago

[deleted]

zkhalique 10y ago

Yeah, the word bias seems to be overloaded with many meanings. I actually wonder if there is any philosophical difference between the word "knowledge" and the word "bias". But about machine learning... a system can learn from a set of outcomes which are the result of a "biased" system, that is to say tainted with an incorrect Bayesian prior which is not properly corrected, such as let's say courts in a racist society or whatever. It would learn the same bias. Because its goal is to maximize compatibility with the outcomes that the humans did. So it perpetuates those weights. The problem is that we don't know whether the human decisions matched the reality. "Did the person commit the crime" for example. We might have to wait until more unbiased estimators for such activity come along, and throw away old historical data. It's sort of like when Black-Scholes became a self-fulfilling prophecy for valuing derivatives, but only after it became widespread. The market started using Black-Scholes to value derivatives, so it became the best model to predict the value of derivatives. But until then, other models might have fit the historical data better.

yummyfajitas 10y ago
Yeah, the word bias seems to be overloaded with many meanings. I actually wonder if there is any philosophical difference between the word "knowledge" and the word "bias". If you interpret the word "bias" statistically - rather than the way innumerate reporters trying to sound profound use it - then it's pretty straightforward to do so. "Knowledge" represents a fixed snapshot of your information about the world. "Bias" represents a tendency for your knowledge to systematically differ from reality even as more data is gathered. In programming terms, knowledge is the data in your DB at a fixe dtime. Bias is a tendency for my inserts to fail while yours succeed. But about machine learning... a system can learn from a set of outcomes which are the result of a "biased" system, that is to say tainted with an incorrect Bayesian prior which is not properly corrected, such as let's say courts in a racist society or whatever. It would learn the same bias. Because its goal is to maximize compatibility with the outcomes that the humans did. So it perpetuates those weights. If the goal of a system is to predict human behaviors, then it will in fact do that. That's not "bias", that's just building a system with the goal of matching human behavior and getting what you asked for. However, if the inputs to your algorithm are biased predictors of the output your outputs can still be unbiased. The tendency of machine learning systems is to detect hidden patterns in the data; biased inputs are just another pattern. I wrote a blog post earlier this year explaining this in detail: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_...

Moshe_Silnorin 10y ago

Given the findings on stereotype accuracy are some of the best-replicated in psychology, I've always found stereotype threat a humorous idea. It implies stereotypes are accurate because they trick us into perpetuating them. Can you think of a more absurd conspiracy theory? It's in keeping with the very popular notion that you can change reality with words, and thus is commonly said to be true in hopes of making it so.

pessimizer 10y ago

> The term "bias" has a real, well-defined meaning in statistics, and both of these terms imply incorrect decisions are being systematically made. Bias is not a word that was invented by statisticians[1], it's a word that statisticians adopted. It's just as legitimate to say that when a machine learning technique is trained with biased input that it produces biased output as it is to say that some particular machine learning method produces biased output from unbiased input. Either way, it's perfectly fine to say that the use of that technique results in bias even if that bias is in an aspect of the output that the algorithm wasn't selected to optimize. It seems that what you're arguing is that the algorithm shouldn't be blamed in these cases, but in addition, you're declaring that there's no such thing as prejudice or bias, and implying that what the algorithms are doing is pulling some (uncomfortable for gatekeepers) platonic truth out of the ether (without any acknowledgement of the possibility of biased input.) Or more simply, you're using this article as an excuse to push a favorite "race realist" agenda which you use to justify a purist libertarianism (and which I agree it requires.) [1] http://etymonline.com/index.php?term=bias http://etymonline.com/index.php?term=bias

yummyfajitas 10y ago
The term "bias" and "prejudice" even outside statistics are defined by the dictionary as being about getting the wrong results: "preconceived opinion that is not based on reason or actual experience". This agrees more or less with the statistical term, it's just less precise. That's also NOT what machine learning algorithms do. If you want to declare an algorithm biased because it optimizes what it was designed to optimize rather than some random tangential goal, then the term has become meaningless. Similarly, my cell phone is broken because it can only make calls and is ineffective at pulling trains. A locomotive is also broken because it doesn't make tacos. It seems that what you're arguing is that the algorithm shouldn't be blamed in these cases, but in addition, you're declaring that there's no such thing as prejudice or bias...what the algorithms are doing is pulling some (uncomfortable for gatekeepers) platonic truth out of the ether (without any acknowledgement of the possibility of biased input.) I have no idea how you could possibly think I made this claim given that I explicitly listed two examples of algorithms which are biased. I then hinted at how algorithms can eliminate this bias. Consider rereading what I wrote. If you want more details on how algorithms correct biased inputs (it's not based on "ether") read the blog post I linked to downthread: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_... Or more simply, you're using this article as an excuse to push a favorite "race realist" agenda which you use to justify a purist libertarianism (and which I agree it requires.) I have no idea why you think "race realist" agendas require purist libertarianism, or vice versa. That's a random political tangent and totally unrelated to this conversation. I'm beginning to think you are seeking to derail the conversation rather than discussing statistics in good faith. In the hopes that I've misunderstood you, I will nevertheless provide you with a good faith clarification. I made no normative claims regarding libertarianism or anything else. The only normative claim I've made is that we should not wrongfully apply an existing term (bias) to completely unrelated concepts (having undesired social outcomes).

throwaway729 10y ago

When there exist two sets of meanings for a word -- one vernacular, the other technical -- we shouldn't try to insist that the technical meaning over-ride the vernacular meaning. That's almost always a losing battle. Insisting that a system isn't biased in the vernacular sense due to a particular technical notion of bias completely misses the point. And we almost never have enough understanding about the world to make a definitive case that the system isn't biased in the vernacular sense.

yummyfajitas 10y ago
My claim is, however, that when discussing statistics we should stick to the statistics terms. If we want to discuss a new concept then we should make up a new word for it. To do otherwise risks having people get confused as to what underlying concepts are actually being discussed. For instance, go read the HN comments here: https://news.ycombinator.com/item?id=13004790 https://news.ycombinator.com/item?id=13004790 Why risk such confusion when we can just use two different words for two different concepts? However, you are also wrong about the vernacular sense, at least if the dictionary is to be believed: Bias: prejudice in favor of or against one thing, person, or group compared with another, usually in a way considered to be unfair. Prejudice: preconceived opinion that is not based on reason or actual experience. This are just informal versions of the statistical term. To translate this to another topic, if we are discussing global warming I don't think we should use the term "warm" to reference "feelings in my heart". Why would we want to confuse whether we are discussing the temperature of the earth or my feelings?
- throwaway729 10y ago
  If this were a conversation about a statistics paper or textbook, or a particular ML algorithm, you might have a point. But we're not discussing statistics. We're discussing the social impact of ML algorithms. > Why risk such confusion when we can just use two different words for two different concepts? The problem with this is that we'll always have that confusion because most people aren't statistics experts. So it's up to the domain experts to know when one meaning or the other is intended. That's true in every domain. > However, you are also wrong about the vernacular sense, at least if the dictionary is to be believed Dictionary definitions don't fully capture the vernacular meaning of words -- especially politically charged words such as "bias". > This are just informal versions of the statistical term. But when real people use the term bias in real conversations, they aren't typically approximating the statistical meaning. Rather, they are typically referring to some a priori ethical limitations on decision making. For example, most people would say that taking skin color into account when approving a housing loan is a form of bias. But they don't mean that in any particularly technical way. There absolutely is moral and political content in the term "bias". For example, when equal lending laws were passed, an unbiased algorithm definitely would've considered race when choosing the outcome of a loan application. African Americans were systematically discriminated against in employment decisions, and often the first to be let go when hard times hit. So a completely unbiased loan officer could come to the statistically valid conclusion that people of color were far more likely to default. None-the-less, a vase majority of people today would agree that the loan officer was exhibiting racial bias. And they're not coming to that conclusion due to innumeracy or some vacuous personal preference. They're perceiving a vast network of causal relationships between decisions that reinforce a systematic bias, even when many of those decisions aren't locally statistically biased > I don't think we should use the term "warm" to reference "feelings in my heart". And I agree, but I agree because non-experts know and understand the difference. And because the term "global warming" with the alternative meaning isn't already in wide-spread vernacular use.

caspianm 10y ago

Say I have two categories, programming articles and non- programming articles, and some other data about each article. And I want to predict whether the article will be interesting or not. And I want to be fair to interesting non-programming articles by having the same proportion of false negatives to correct positives in the non-programming subset of articles as in the programming subset of articles. Is there a technical term for that in statistics? It's like trying to get a representative sample, but only representative in one specific way (topic), and deliberately non representative in another (interestingness) I think this could get at one of the things people mean, and it might be interesting to see how this trades off against overall accuracy or representativeness in other categories.