4 ms·

For starters, any machine learning system that relies on word embeddings learned from biased corpora to make predictions (or to make decisions!) will necessaril

For starters, any machine learning system that relies on word embeddings learned from biased corpora to make predictions (or to make decisions!) will necessarily be biased in favor of certain groups of people and against others. This is not true. Here's an oversimplified example. Suppose your machine learning system wants to predict something, e.g. loan repayment probabilities. One input might be a written evaluation by a loan officer. When trained on a corpora of group X, the predicted probability might be: pred = a*written_evaluation + other_factors (Using linear regression to make example simple.) However, now lets suppose the written evaluation is biased to the tune of 25% against group Y. I.e., group Y has written scores that are 25% less than group X. Then a new predictor which includes pairwise terms, trained on a corpora of group X and Y, will work out to be: pred = a*written_evaluation + 0.33*written_evaluation*isY + other_factors This predictor would be unbiased. In general, if you have a biased input and the biasing factor is also present in your input, your model should correct the bias. (Obvious caveats: your model needs to be sufficiently expressive, etc.) Interestingly, everyone's favorite bogeyman, namely redundant encoding ( http://deliprao.com/archives/129 http://deliprao.com/archives/129 ) will actually help fix this problem *even if you don't include the biasing factors in the model.

cs702 10y ago
...now lets suppose the written evaluation is biased to the tune of 25% against group Y... How do you find out that the written evaluation is biased "to the tune of 25% against group Y?" THAT is the problem. It's not obvious to me how you would go about determining written evaluations are biased (and to what extent!) against group Y without somehow relying on subjective societal values that are different everywhere and continually evolving.
- yummyfajitas 10y ago
  Finding out is the easy part. I don't mean to trivialize it, because doing stats right is actually a very technical matter, but this is just ordinary statistics. You build a sufficiently expressive statistical model and include the potentially biasing factors as features in the model. Then the model will correct the bias all by itself because correcting for bias maximizes accuracy. In the example above, you find the bias by doing linear regression and including (written_evaluation x isY) as a term. Least squares will handle the rest. If you using something fancier than least squares (e.g. deep neural networks, SVMs with interesting kernels), you probably don't even need to explicitly include potentially debiasing terms - the model will do it for you. I give toy examples (designed to illustrate the point and also be easy to understand) here: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_... This paper does the same thing - it discovers that standard predictors of college performance (grades, GPA) are biased in favor of blacks and men, against Asians and women, and the model itself fixes these biases: http://ftp.iza.org/dp8733.pdf http://ftp.iza.org/dp8733.pdf Statistics turns fixing racism into a math problem. If the topic were anything less emotionally charged, you wouldn't even think twice about it. If I suggested including `isMobile`, `isDesktop` and `isTablet` as features in an ad-targeting algorithm to deal with the fact that users on mobile and desktop browse differently, you'd yawn.
  - cs702 10y ago
    ...include the potentially biasing factors as features... Who decides what the "potentially biasing factors" are? How is that decided without somehow relying on subjective societal values? Factors that no one thought were biased in the past are considered biased today; factors that no one thinks are biased today may be considered biased in the future; and factors that you and I consider biased today may not be considered biased by people in other parts of the world. I don't know how one would go about finding those "potentially biasing factors" without relying on subjective societal values that are different everywhere and always evolving.
    - yummyfajitas 10y ago
      A potentially biasing factor is a factor that you think would be predictive if you included it in the model. If it's actually predictive, you win, your model becomes more accurate and you make more money. Go read the wikipedia article on the topic: https://en.wikipedia.org/wiki/Omitted-variable_bias https://en.wikipedia.org/wiki/Omitted-variable_bias It's true that as we learn more things we discover new predictive factors. That doesn't make them subjective. A lung cancer model that excludes smoking is not subjective, it's just wrong. And the way to fix the model is to add smoking as a feature and re-run your regression. Again, would you make the same argument you just made if I said I had an accurate ad-targeting model?
      - cs702 10y ago
        OK, I see where the disconnect is. I think the best way to address it is with an example. Many people today would object a priori to businesses using race as a factor to predict loan default risk, regardless of whether doing that makes the predictions more accurate or not. In many cases, using race as a factor WILL get you in trouble with the law (e.g., redlining is illegal in the US). Please tell me, how would you predict what factors society will find objectionable in the future (like race today)?
        yummyfajitas 10y ago

cs702 10y ago

...now lets suppose the written evaluation is biased to the tune of 25% against group Y... How do you find out that the written evaluation is biased "to the tune of 25% against group Y?" THAT is the problem. It's not obvious to me how you would go about determining written evaluations are biased (and to what extent!) against group Y without somehow relying on subjective societal values that are different everywhere and continually evolving.

yummyfajitas 10y ago
Finding out is the easy part. I don't mean to trivialize it, because doing stats right is actually a very technical matter, but this is just ordinary statistics. You build a sufficiently expressive statistical model and include the potentially biasing factors as features in the model. Then the model will correct the bias all by itself because correcting for bias maximizes accuracy. In the example above, you find the bias by doing linear regression and including (written_evaluation x isY) as a term. Least squares will handle the rest. If you using something fancier than least squares (e.g. deep neural networks, SVMs with interesting kernels), you probably don't even need to explicitly include potentially debiasing terms - the model will do it for you. I give toy examples (designed to illustrate the point and also be easy to understand) here: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_... This paper does the same thing - it discovers that standard predictors of college performance (grades, GPA) are biased in favor of blacks and men, against Asians and women, and the model itself fixes these biases: http://ftp.iza.org/dp8733.pdf http://ftp.iza.org/dp8733.pdf Statistics turns fixing racism into a math problem. If the topic were anything less emotionally charged, you wouldn't even think twice about it. If I suggested including `isMobile`, `isDesktop` and `isTablet` as features in an ad-targeting algorithm to deal with the fact that users on mobile and desktop browse differently, you'd yawn.
- cs702 10y ago
  ...include the potentially biasing factors as features... Who decides what the "potentially biasing factors" are? How is that decided without somehow relying on subjective societal values? Factors that no one thought were biased in the past are considered biased today; factors that no one thinks are biased today may be considered biased in the future; and factors that you and I consider biased today may not be considered biased by people in other parts of the world. I don't know how one would go about finding those "potentially biasing factors" without relying on subjective societal values that are different everywhere and always evolving.
  - yummyfajitas 10y ago
    A potentially biasing factor is a factor that you think would be predictive if you included it in the model. If it's actually predictive, you win, your model becomes more accurate and you make more money. Go read the wikipedia article on the topic: https://en.wikipedia.org/wiki/Omitted-variable_bias https://en.wikipedia.org/wiki/Omitted-variable_bias It's true that as we learn more things we discover new predictive factors. That doesn't make them subjective. A lung cancer model that excludes smoking is not subjective, it's just wrong. And the way to fix the model is to add smoking as a feature and re-run your regression. Again, would you make the same argument you just made if I said I had an accurate ad-targeting model?
    - cs702 10y ago
      OK, I see where the disconnect is. I think the best way to address it is with an example. Many people today would object a priori to businesses using race as a factor to predict loan default risk, regardless of whether doing that makes the predictions more accurate or not. In many cases, using race as a factor WILL get you in trouble with the law (e.g., redlining is illegal in the US). Please tell me, how would you predict what factors society will find objectionable in the future (like race today)?
      - yummyfajitas 10y ago