4 ms·

Doing this was a great idea. Great paper: easy-to-follow and to-the-point. The results are not too surprising, as the models for learning word embeddings like

by cs702 10y ago

Doing this was a great idea. Great paper: easy-to-follow and to-the-point. The results are not too surprising, as the models for learning word embeddings like GloVe, word2vec, etc. learn to map to vectors existing relationships between words in training corpora. If a corpus is biased, the embeddings learned from it will necessarily be biased too. However, the implications of this finding are wide-ranging. For starters, any machine learning system that relies on word embeddings learned from biased corpora to make predictions (or to make decisions!) will necessarily be biased in favor of certain groups of people and against others. Moreover, it's not obvious to me how one would go about obtaining "unbiased" corpora without somehow relying on subjective societal values that are different everywhere and continually evolving. You have raised an important, non-trivial problem.

wolfgke 10y ago
> Moreover, it's not obvious to me how one would go about obtaining "unbiased" corpora without somehow relying on subjective societal values that are different everywhere and continually evolving. I don't believe that problem will ever be completely solvable. But I think the road to go is to make these assumptions always explicit. I.e. when the machine learning system derives a result, program it to additionally return a proof of how it came to this result. And also give a way to let the ML system return a list of all axioms and derivation rules that it has currently learned, so that they can independently be checked how much they are biased and can thus be corrected.
- nl 10y ago
  It's pretty hard to return "rules" for a ML system, especially a non-linear system. Google is currently working systems that use a trillion features - I can't imagine returning some kind of rule list for that. LIME[1] is a nice start, though. [1] https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime https://www.oreilly.com/learning/introduction-to-local-inter...
  - wolfgke 10y ago
    Another text about interpreting convolutional neural networks: http://cs231n.github.io/understanding-cnn/ http://cs231n.github.io/understanding-cnn/ > Google is currently working systems that use a trillion features - I can't imagine returning some kind of rule list for that. As I wrote: It would already help if the ML system as a first step returned the derivation with only the rules that were concretely used for a concrete derivation - this list is much shorter and can thus much easier be checked.
yummyfajitas 10y ago
For starters, any machine learning system that relies on word embeddings learned from biased corpora to make predictions (or to make decisions!) will necessarily be biased in favor of certain groups of people and against others. This is not true. Here's an oversimplified example. Suppose your machine learning system wants to predict something, e.g. loan repayment probabilities. One input might be a written evaluation by a loan officer. When trained on a corpora of group X, the predicted probability might be: pred = a*written_evaluation + other_factors (Using linear regression to make example simple.) However, now lets suppose the written evaluation is biased to the tune of 25% against group Y. I.e., group Y has written scores that are 25% less than group X. Then a new predictor which includes pairwise terms, trained on a corpora of group X and Y, will work out to be: pred = a*written_evaluation + 0.33*written_evaluation*isY + other_factors This predictor would be unbiased. In general, if you have a biased input and the biasing factor is also present in your input, your model should correct the bias. (Obvious caveats: your model needs to be sufficiently expressive, etc.) Interestingly, everyone's favorite bogeyman, namely redundant encoding ( http://deliprao.com/archives/129 http://deliprao.com/archives/129 ) will actually help fix this problem *even if you don't include the biasing factors in the model.
- cs702 10y ago
  ...now lets suppose the written evaluation is biased to the tune of 25% against group Y... How do you find out that the written evaluation is biased "to the tune of 25% against group Y?" THAT is the problem. It's not obvious to me how you would go about determining written evaluations are biased (and to what extent!) against group Y without somehow relying on subjective societal values that are different everywhere and continually evolving.
  - yummyfajitas 10y ago
    Finding out is the easy part. I don't mean to trivialize it, because doing stats right is actually a very technical matter, but this is just ordinary statistics. You build a sufficiently expressive statistical model and include the potentially biasing factors as features in the model. Then the model will correct the bias all by itself because correcting for bias maximizes accuracy. In the example above, you find the bias by doing linear regression and including (written_evaluation x isY) as a term. Least squares will handle the rest. If you using something fancier than least squares (e.g. deep neural networks, SVMs with interesting kernels), you probably don't even need to explicitly include potentially debiasing terms - the model will do it for you. I give toy examples (designed to illustrate the point and also be easy to understand) here: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_... This paper does the same thing - it discovers that standard predictors of college performance (grades, GPA) are biased in favor of blacks and men, against Asians and women, and the model itself fixes these biases: http://ftp.iza.org/dp8733.pdf http://ftp.iza.org/dp8733.pdf Statistics turns fixing racism into a math problem. If the topic were anything less emotionally charged, you wouldn't even think twice about it. If I suggested including `isMobile`, `isDesktop` and `isTablet` as features in an ad-targeting algorithm to deal with the fact that users on mobile and desktop browse differently, you'd yawn.

by cs702 10y ago

wolfgke 10y ago

> Moreover, it's not obvious to me how one would go about obtaining "unbiased" corpora without somehow relying on subjective societal values that are different everywhere and continually evolving. I don't believe that problem will ever be completely solvable. But I think the road to go is to make these assumptions always explicit. I.e. when the machine learning system derives a result, program it to additionally return a proof of how it came to this result. And also give a way to let the ML system return a list of all axioms and derivation rules that it has currently learned, so that they can independently be checked how much they are biased and can thus be corrected.

nl 10y ago
It's pretty hard to return "rules" for a ML system, especially a non-linear system. Google is currently working systems that use a trillion features - I can't imagine returning some kind of rule list for that. LIME[1] is a nice start, though. [1] https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime https://www.oreilly.com/learning/introduction-to-local-inter...
- wolfgke 10y ago
  Another text about interpreting convolutional neural networks: http://cs231n.github.io/understanding-cnn/ http://cs231n.github.io/understanding-cnn/ > Google is currently working systems that use a trillion features - I can't imagine returning some kind of rule list for that. As I wrote: It would already help if the ML system as a first step returned the derivation with only the rules that were concretely used for a concrete derivation - this list is much shorter and can thus much easier be checked.

yummyfajitas 10y ago

For starters, any machine learning system that relies on word embeddings learned from biased corpora to make predictions (or to make decisions!) will necessarily be biased in favor of certain groups of people and against others. This is not true. Here's an oversimplified example. Suppose your machine learning system wants to predict something, e.g. loan repayment probabilities. One input might be a written evaluation by a loan officer. When trained on a corpora of group X, the predicted probability might be: pred = a*written_evaluation + other_factors (Using linear regression to make example simple.) However, now lets suppose the written evaluation is biased to the tune of 25% against group Y. I.e., group Y has written scores that are 25% less than group X. Then a new predictor which includes pairwise terms, trained on a corpora of group X and Y, will work out to be: pred = a*written_evaluation + 0.33*written_evaluation*isY + other_factors This predictor would be unbiased. In general, if you have a biased input and the biasing factor is also present in your input, your model should correct the bias. (Obvious caveats: your model needs to be sufficiently expressive, etc.) Interestingly, everyone's favorite bogeyman, namely redundant encoding ( http://deliprao.com/archives/129 http://deliprao.com/archives/129 ) will actually help fix this problem *even if you don't include the biasing factors in the model.

cs702 10y ago
...now lets suppose the written evaluation is biased to the tune of 25% against group Y... How do you find out that the written evaluation is biased "to the tune of 25% against group Y?" THAT is the problem. It's not obvious to me how you would go about determining written evaluations are biased (and to what extent!) against group Y without somehow relying on subjective societal values that are different everywhere and continually evolving.
- yummyfajitas 10y ago
  Finding out is the easy part. I don't mean to trivialize it, because doing stats right is actually a very technical matter, but this is just ordinary statistics. You build a sufficiently expressive statistical model and include the potentially biasing factors as features in the model. Then the model will correct the bias all by itself because correcting for bias maximizes accuracy. In the example above, you find the bias by doing linear regression and including (written_evaluation x isY) as a term. Least squares will handle the rest. If you using something fancier than least squares (e.g. deep neural networks, SVMs with interesting kernels), you probably don't even need to explicitly include potentially debiasing terms - the model will do it for you. I give toy examples (designed to illustrate the point and also be easy to understand) here: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_... This paper does the same thing - it discovers that standard predictors of college performance (grades, GPA) are biased in favor of blacks and men, against Asians and women, and the model itself fixes these biases: http://ftp.iza.org/dp8733.pdf http://ftp.iza.org/dp8733.pdf Statistics turns fixing racism into a math problem. If the topic were anything less emotionally charged, you wouldn't even think twice about it. If I suggested including `isMobile`, `isDesktop` and `isTablet` as features in an ad-targeting algorithm to deal with the fact that users on mobile and desktop browse differently, you'd yawn.