9 ms·

Language contains human biases, and so will machines trained on language corpora

124 points by r721 10y ago

jonbarker 10y ago
This is related to the alphago series. Human language describing expert intuition created norms about the game that became "standard expert practice" but not truly efficient play.
skybrian 10y ago
Here's an example of how to remove bias from a dataset: https://socialmediacollective.org/2016/08/06/amplifying-and-neutralizing-gender-bias-using-machine-learning-algorithms/ https://socialmediacollective.org/2016/08/06/amplifying-and-...
- randomwalker 10y ago
  OP here. We address this argument in detail in our paper, and we're deeply skeptical of it. See the sections titled "Challenges in addressing bias" and "Awareness is better than blindness". Here's the short version: We view the approach of "debiasing" word embeddings (Bolukbasi et al., 2016) with skepticism. If we view AI as perception followed by action, debiasing alters the AI’s perception (and model) of the world, rather than how it acts on that perception. This gives the AI an incomplete understanding of the world. We see debiasing as "fairness through blindness". It has its place, but also important limits: prejudice can creep back in through proxies (although we should note that Bolukbasi et al. (2016) do consider "indirect bias" in their paper). Efforts to fight prejudice at the level of the initial representation will necessarily hurt meaning and accuracy, and will themselves be hard to adapt as societal understanding of fairness evolves Direct link to our paper: http://randomwalker.info/publications/language-bias.pdf http://randomwalker.info/publications/language-bias.pdf
  - rspeer 10y ago
    I agree with the suggestion to de-bias the application and not the representation itself. Recently I was using a version of Conceptnet Numberbatch (word embeddings built from ConceptNet, word2vec, and GloVe data that perform very well on evaluations) as an input to sentiment analysis. So its input happens to include a crawl of the Web (via GloVe) and things that came to mind as people played word games (via ConceptNet). All of this went into a straightforward support vector regression with AFINN as training data. You can probably see where this is going. The resulting sentiment classification of words such as "Mexican", "Chinese", and "black" would make Donald Trump blush. I think the current version is less extreme about it, but there is still an effect to be corrected: it ends up with slightly negative opinions about most words that describe groups of people, especially the more dissimilar they are from the American majority. So my correction is to add words about groups of people to the training data for the sentiment analyzer, with a lot of weight, saying that their output has to be 0.
  - foolrush 10y ago
    Are there any overlaps or thoughts regarding Sapir Whorf here in your research?
    - randomwalker 10y ago
      Yes! We address this in the section "Implications for understanding human prejudice". The simplicity and strength of our results suggests a new null hypothesis for explaining origins of prejudicial behavior in humans, namely, the implicit transmission of ingroup/outgroup identity information through language. That is, before providing an explicit or institutional explanation for why individuals make decisions that disadvantage one group with regards to another, one must show that the unjust decision was not a simple outcome of unthinking reproduction of statistical regularities absorbed with language. Similarly, before positing complex models for how prejudicial attitudes perpetuate from one generation to the next or from one group to another, we must check whether simply learning language is sufficient to explain the observed transmission of prejudice. These new null hypotheses are important not because we necessarily expect them to be true in most cases, but because Occam’s razor now requires that we eliminate them, or at least quantify findings about prejudice in comparison to what is explainable from language transmission alone (The paper has more along these lines.)
  - deleted 10y ago
    [deleted]
  - jdp23 10y ago
    I'm not convinced by your skepticism about correcting prejudiced bias. Debiasing certainly gives the AI a understanding of the world than the original (biased) language dataset, but it's not necessarily less complete - or less accurate. After all, any one corpus is incomplete, and has biases based on the items that were chosen for it - which are likely to reflect the biases of the past, and of the person choosing the corpus. It may not be a "complete" or "accurate" reflection of today's world - let alone the future. So it's not at all clear to me that efforts to undo the bias will necessarily make it less "accurate".
    - AnthonyMouse 10y ago
      > Debiasing certainly gives the AI a understanding of the world than the original (biased) language dataset, but it's not necessarily less complete - or less accurate. If you're translating from an ungendered language and have to choose, the only way you're going to get anything sensible is from context and common usage. Which is going to choose "she is a nurse" because an algorithm that can deduce that fathers are most likely male can also deduce that nurses are most likely female. But without that you get bad translations like "she is a father" and "he is a fine ship" and "John is her own person."
      - yummyfajitas 10y ago
        "She is a nurse" is also not a bias. It's a prior and a valid one - the system will be right 93% of the time. http://work.chron.com/gender-equality-issues-nursing-careers-11250.html http://work.chron.com/gender-equality-issues-nursing-careers... A bias would be if it incorrectly weighted "JOHN" and "nurse", and used the feminine for "John the nurse".
        jdp23 10y ago
        > "She is a nurse" is also not a bias. It's a prior ... Assuming that lower-status professions are female and higher-status professions are male ("he is a doctor") when translating ungendered words is indeed a bias. > the system will be right 93% of the time. And "this person is a doctor, that person is a nurse" will be right 100% of the time.
  - igravious 10y ago
    Interesting paper, thanks! See how you sway the argument in your favour using words with negative connotations like "fairness through blindness" and "hurt meaning and accuracy". Nobody would want to deliberately blind or hurt something, would they? How about rebalance or recalibrate or re-correct. A concrete analogy: 1) I have a meter measuring stick but I discover that it was made wrong, it is actually 2mm shorter than advertised. Every time I make a measurement with it I have to add 2mm to the measurement. Would it not be better to use a more accurate stick and not have to continually compensate?
    - im4w1l 10y ago
      Stereotypes have great predictive power. The reason we sometimes avoid them is they can lead to outcomes that are seen as undesirable.
    - pixl97 10y ago
      With your analogy that would assume we know exactly how long a meter is. "We know that we are wrong, but we don't know the exact right answer". Also language shifts and biases are not constants. Oh, then you have the issue of a corpus attempting to manipulate the learning algorithm itself.
randomwalker 10y ago
Coauthor here. The blog post is written in relatively non-technical language for a general audience, but our paper has tons of technical details that HN readers might enjoy. Give it a read! http://randomwalker.info/publications/language-bias.pdf http://randomwalker.info/publications/language-bias.pdf
- cs702 10y ago
  Doing this was a great idea. Great paper: easy-to-follow and to-the-point. The results are not too surprising, as the models for learning word embeddings like GloVe, word2vec, etc. learn to map to vectors existing relationships between words in training corpora. If a corpus is biased, the embeddings learned from it will necessarily be biased too. However, the implications of this finding are wide-ranging. For starters, any machine learning system that relies on word embeddings learned from biased corpora to make predictions (or to make decisions!) will necessarily be biased in favor of certain groups of people and against others. Moreover, it's not obvious to me how one would go about obtaining "unbiased" corpora without somehow relying on subjective societal values that are different everywhere and continually evolving. You have raised an important, non-trivial problem.
  - wolfgke 10y ago
    > Moreover, it's not obvious to me how one would go about obtaining "unbiased" corpora without somehow relying on subjective societal values that are different everywhere and continually evolving. I don't believe that problem will ever be completely solvable. But I think the road to go is to make these assumptions always explicit. I.e. when the machine learning system derives a result, program it to additionally return a proof of how it came to this result. And also give a way to let the ML system return a list of all axioms and derivation rules that it has currently learned, so that they can independently be checked how much they are biased and can thus be corrected.
    - nl 10y ago
      It's pretty hard to return "rules" for a ML system, especially a non-linear system. Google is currently working systems that use a trillion features - I can't imagine returning some kind of rule list for that. LIME[1] is a nice start, though. [1] https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime https://www.oreilly.com/learning/introduction-to-local-inter...
protomyth 10y ago
Did you research focus on English or does the nearly the same biases hold for all modern languages? I would love to see a competently trained person try this with the Asian and Native American languages. I dearly hope we will let the bias "humanity is good, don't kill us" stay in.
- Terribledactyl 10y ago
  My German professor mentioned a few times in class about how automated translators had picked up sexist biases because of older training materials.
  - wolfgke 10y ago
    Which is to me rather a sign of how society changes meaning of the language to indoctrinate a kind of social propaganda. Greetings from George Orwell's 1984.
    - throwanem 10y ago
      In fairness, there's a distinction to be made here between directed and undirected change.
      - wolfgke 10y ago
        When SJW warrior groups make campaigns that from now on some list of words has to be considered as racist, sexist, ...ist etc. I think one can talk of directed change.
        akhilcacharya 10y ago
        ..what if those words are?
        wolfgke 10y ago
        At least people before did not consider them to be. This shows that proving that say are is not so easy.
mc32 10y ago
There are many instances where bias is not a desired result. We rather not preassociate negative denoting words with a generally good set of people. On the other hand, there ate situations where we want to have bias. A bias against unjust discrimination, a bias against psychopaths, a bias against repeat offenders, etc. So to me, there are places where we want to retain bias and places we want to rid bias and that will be up to society to decide, as they decide everyday laws and conduct, etc.
jcoffland 10y ago
This whole concept makes the silly assumption that today's machine learning results in machine understanding. Without understanding human biases, as they are embedded in language, are irrelevant.
- jhanschoo 10y ago
  That's a bold, ill-defined, and unsubstantiated claim. Leaving machine understanding undefined, I can think of an example in which these biases are relevant. For example, if you're offering a search engine solution incorporating clustering, you'd be concerned that your search retrievals aren't associating racial minorities with pejorative prejudices.
  - wolfgke 10y ago
    > For example, if you're offering a search engine solution incorporating clustering, you'd be concerned that your search retrievals aren't associating racial minorities with pejorative prejudices. If in the data racial minorities are associated with pejorative prejudices, this is plain distorting of the truth. If you are concerned about this associations, don't shoot the messenger, but the people writing the original texts.
    - jhanschoo 10y ago
      > this is plain distorting of the truth. In the context of business and many other interests, it's expedient to present a more polite face. You're providing a service, not a mirror. In the context of a search engine service, say, a specialized, domain-specific search for a certain profession, such biases often distract and detract from the quality of the service you provide to your customers.
troels 10y ago
"Instead, we suggest that mitigating prejudice should be a separate component of an AI system." Much like I imagine we humans deals with the concept.
deleted 10y ago
[deleted]
nix0n 10y ago
Similarly, algorithms trained to mimic human decisions will mimic human biases. See https://news.ycombinator.com/item?id=11753805 https://news.ycombinator.com/item?id=11753805 (probably others too)
- yummyfajitas 10y ago
  That article is fundamentally dishonest. The author's own statistical analysis (see their R-script and my comments on the article) cannot reject the null hypothesis that the algorithm is unbiased. I wrote a more detailed critique here: https://www.chrisstucchio.com/blog/2016/propublica_is_lying.html https://www.chrisstucchio.com/blog/2016/propublica_is_lying....
yummyfajitas 10y ago
There was a great comment here discussing the Pathetic fallacy, and how we humans apply it to machine learning systems. (It's now deleted and I don't know why.) Specifically, we treat machine learning systems as anthropomorphic, and assume they will reproduce our biases. https://en.wikipedia.org/wiki/Pathetic_fallacy https://en.wikipedia.org/wiki/Pathetic_fallacy In reality, ML systems will generally correct human biases. And by bias, I really do mean bias in the statistical sense - systematically getting things wrong in a particular direction. Now this article does a great job of explaining how ML systems might understand the meaning of words, and that meaning may contain bias. However, such a system is merely an input into a separate system which actually makes decisions based on those inputs. Extracting meaning from text makes no decisions of it's own. If that later system wants to make accurate decisions, then the best way to do that is to correct for the aforementioned bias, assuming that bias is really bias as opposed to just a correct but undesirable belief about the world [1]. I wrote a blog post a while back that goes into this idea with a bit more math, and which demonstrates some real world "learning" algorithms (mostly linear regression) actually correcting biases: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_...
- wyager 10y ago
  One problem you will run into here is that political use of the word "bias" isn't any sort of statistical claim; it's often just used to mean that something violates vague social expectations about what information is "acceptable" to use when making decisions. ML algorithms don't care; they will make the best possible decision based on the information they have, even if their though process (so to speak) is "biased" in the political sense that it may use gender, race, nationality, etc. to help make optimal decisions.
  - yummyfajitas 10y ago
    Yes, unfortunately the term is overloaded. "Bias" can also mean "making correct decisions that I wish were incorrect". That's why I explicitly defined "bias". In the past era, e.g. 1980-2010, it was possible to use vague emotive language to support all kinds of disparate things. As a concrete example that I touch on in my post, and since racism is the loaded undercurrent of this example, we like to pretend that eliminating racial or sexual bias (in the sense of making wrong decisions) will get us proportional representation. Algorithms are bringing us to an age where analytic philosophy is becoming really important. You can tell an algorithm to give you proportional representation, or you can tell it to be racially/sexually unbiased. But the algorithm will reflect reality and reality may not agree with your assumptions; you can't assume that asking for one will give you the other. So now we get into trolley problems: how much meritocracy/equal opportunity will you sacrifice to get proportional representation? Unlike before, this is now a choice you need to explicitly and openly state and acknowledge.
shanacarp 10y ago
Can you measure linguistic shifts with your model?

124 points by r721 10y ago

This is related to the alphago series. Human language describing expert intuition created norms about the game that became "standard expert practice" but not truly efficient play.

skybrian 10y ago

Here's an example of how to remove bias from a dataset: https://socialmediacollective.org/2016/08/06/amplifying-and-neutralizing-gender-bias-using-machine-learning-algorithms/ https://socialmediacollective.org/2016/08/06/amplifying-and-...

randomwalker 10y ago
OP here. We address this argument in detail in our paper, and we're deeply skeptical of it. See the sections titled "Challenges in addressing bias" and "Awareness is better than blindness". Here's the short version: We view the approach of "debiasing" word embeddings (Bolukbasi et al., 2016) with skepticism. If we view AI as perception followed by action, debiasing alters the AI’s perception (and model) of the world, rather than how it acts on that perception. This gives the AI an incomplete understanding of the world. We see debiasing as "fairness through blindness". It has its place, but also important limits: prejudice can creep back in through proxies (although we should note that Bolukbasi et al. (2016) do consider "indirect bias" in their paper). Efforts to fight prejudice at the level of the initial representation will necessarily hurt meaning and accuracy, and will themselves be hard to adapt as societal understanding of fairness evolves Direct link to our paper: http://randomwalker.info/publications/language-bias.pdf http://randomwalker.info/publications/language-bias.pdf
- rspeer 10y ago
  I agree with the suggestion to de-bias the application and not the representation itself. Recently I was using a version of Conceptnet Numberbatch (word embeddings built from ConceptNet, word2vec, and GloVe data that perform very well on evaluations) as an input to sentiment analysis. So its input happens to include a crawl of the Web (via GloVe) and things that came to mind as people played word games (via ConceptNet). All of this went into a straightforward support vector regression with AFINN as training data. You can probably see where this is going. The resulting sentiment classification of words such as "Mexican", "Chinese", and "black" would make Donald Trump blush. I think the current version is less extreme about it, but there is still an effect to be corrected: it ends up with slightly negative opinions about most words that describe groups of people, especially the more dissimilar they are from the American majority. So my correction is to add words about groups of people to the training data for the sentiment analyzer, with a lot of weight, saying that their output has to be 0.
- foolrush 10y ago
  Are there any overlaps or thoughts regarding Sapir Whorf here in your research?
  - randomwalker 10y ago
    Yes! We address this in the section "Implications for understanding human prejudice". The simplicity and strength of our results suggests a new null hypothesis for explaining origins of prejudicial behavior in humans, namely, the implicit transmission of ingroup/outgroup identity information through language. That is, before providing an explicit or institutional explanation for why individuals make decisions that disadvantage one group with regards to another, one must show that the unjust decision was not a simple outcome of unthinking reproduction of statistical regularities absorbed with language. Similarly, before positing complex models for how prejudicial attitudes perpetuate from one generation to the next or from one group to another, we must check whether simply learning language is sufficient to explain the observed transmission of prejudice. These new null hypotheses are important not because we necessarily expect them to be true in most cases, but because Occam’s razor now requires that we eliminate them, or at least quantify findings about prejudice in comparison to what is explainable from language transmission alone (The paper has more along these lines.)
- deleted 10y ago
  [deleted]
- jdp23 10y ago
  I'm not convinced by your skepticism about correcting prejudiced bias. Debiasing certainly gives the AI a understanding of the world than the original (biased) language dataset, but it's not necessarily less complete - or less accurate. After all, any one corpus is incomplete, and has biases based on the items that were chosen for it - which are likely to reflect the biases of the past, and of the person choosing the corpus. It may not be a "complete" or "accurate" reflection of today's world - let alone the future. So it's not at all clear to me that efforts to undo the bias will necessarily make it less "accurate".
  - AnthonyMouse 10y ago
    > Debiasing certainly gives the AI a understanding of the world than the original (biased) language dataset, but it's not necessarily less complete - or less accurate. If you're translating from an ungendered language and have to choose, the only way you're going to get anything sensible is from context and common usage. Which is going to choose "she is a nurse" because an algorithm that can deduce that fathers are most likely male can also deduce that nurses are most likely female. But without that you get bad translations like "she is a father" and "he is a fine ship" and "John is her own person."
    - yummyfajitas 10y ago
      "She is a nurse" is also not a bias. It's a prior and a valid one - the system will be right 93% of the time. http://work.chron.com/gender-equality-issues-nursing-careers-11250.html http://work.chron.com/gender-equality-issues-nursing-careers... A bias would be if it incorrectly weighted "JOHN" and "nurse", and used the feminine for "John the nurse".
      - jdp23 10y ago
        > "She is a nurse" is also not a bias. It's a prior ... Assuming that lower-status professions are female and higher-status professions are male ("he is a doctor") when translating ungendered words is indeed a bias. > the system will be right 93% of the time. And "this person is a doctor, that person is a nurse" will be right 100% of the time.
- igravious 10y ago
  Interesting paper, thanks! See how you sway the argument in your favour using words with negative connotations like "fairness through blindness" and "hurt meaning and accuracy". Nobody would want to deliberately blind or hurt something, would they? How about rebalance or recalibrate or re-correct. A concrete analogy: 1) I have a meter measuring stick but I discover that it was made wrong, it is actually 2mm shorter than advertised. Every time I make a measurement with it I have to add 2mm to the measurement. Would it not be better to use a more accurate stick and not have to continually compensate?
  - im4w1l 10y ago
    Stereotypes have great predictive power. The reason we sometimes avoid them is they can lead to outcomes that are seen as undesirable.
  - pixl97 10y ago
    With your analogy that would assume we know exactly how long a meter is. "We know that we are wrong, but we don't know the exact right answer". Also language shifts and biases are not constants. Oh, then you have the issue of a corpus attempting to manipulate the learning algorithm itself.

randomwalker 10y ago

Coauthor here. The blog post is written in relatively non-technical language for a general audience, but our paper has tons of technical details that HN readers might enjoy. Give it a read! http://randomwalker.info/publications/language-bias.pdf http://randomwalker.info/publications/language-bias.pdf

cs702 10y ago
Doing this was a great idea. Great paper: easy-to-follow and to-the-point. The results are not too surprising, as the models for learning word embeddings like GloVe, word2vec, etc. learn to map to vectors existing relationships between words in training corpora. If a corpus is biased, the embeddings learned from it will necessarily be biased too. However, the implications of this finding are wide-ranging. For starters, any machine learning system that relies on word embeddings learned from biased corpora to make predictions (or to make decisions!) will necessarily be biased in favor of certain groups of people and against others. Moreover, it's not obvious to me how one would go about obtaining "unbiased" corpora without somehow relying on subjective societal values that are different everywhere and continually evolving. You have raised an important, non-trivial problem.
- wolfgke 10y ago
  > Moreover, it's not obvious to me how one would go about obtaining "unbiased" corpora without somehow relying on subjective societal values that are different everywhere and continually evolving. I don't believe that problem will ever be completely solvable. But I think the road to go is to make these assumptions always explicit. I.e. when the machine learning system derives a result, program it to additionally return a proof of how it came to this result. And also give a way to let the ML system return a list of all axioms and derivation rules that it has currently learned, so that they can independently be checked how much they are biased and can thus be corrected.
  - nl 10y ago
    It's pretty hard to return "rules" for a ML system, especially a non-linear system. Google is currently working systems that use a trillion features - I can't imagine returning some kind of rule list for that. LIME[1] is a nice start, though. [1] https://www.oreilly.com/learning/introduction-to-local-interpretable-model-agnostic-explanations-lime https://www.oreilly.com/learning/introduction-to-local-inter...

protomyth 10y ago

Did you research focus on English or does the nearly the same biases hold for all modern languages? I would love to see a competently trained person try this with the Asian and Native American languages. I dearly hope we will let the bias "humanity is good, don't kill us" stay in.

Terribledactyl 10y ago
My German professor mentioned a few times in class about how automated translators had picked up sexist biases because of older training materials.
- wolfgke 10y ago
  Which is to me rather a sign of how society changes meaning of the language to indoctrinate a kind of social propaganda. Greetings from George Orwell's 1984.
  - throwanem 10y ago
    In fairness, there's a distinction to be made here between directed and undirected change.
    - wolfgke 10y ago
      When SJW warrior groups make campaigns that from now on some list of words has to be considered as racist, sexist, ...ist etc. I think one can talk of directed change.
      - akhilcacharya 10y ago
        ..what if those words are?
        wolfgke 10y ago
        At least people before did not consider them to be. This shows that proving that say are is not so easy.

mc32 10y ago

There are many instances where bias is not a desired result. We rather not preassociate negative denoting words with a generally good set of people. On the other hand, there ate situations where we want to have bias. A bias against unjust discrimination, a bias against psychopaths, a bias against repeat offenders, etc. So to me, there are places where we want to retain bias and places we want to rid bias and that will be up to society to decide, as they decide everyday laws and conduct, etc.

jcoffland 10y ago

This whole concept makes the silly assumption that today's machine learning results in machine understanding. Without understanding human biases, as they are embedded in language, are irrelevant.

jhanschoo 10y ago
That's a bold, ill-defined, and unsubstantiated claim. Leaving machine understanding undefined, I can think of an example in which these biases are relevant. For example, if you're offering a search engine solution incorporating clustering, you'd be concerned that your search retrievals aren't associating racial minorities with pejorative prejudices.
- wolfgke 10y ago
  > For example, if you're offering a search engine solution incorporating clustering, you'd be concerned that your search retrievals aren't associating racial minorities with pejorative prejudices. If in the data racial minorities are associated with pejorative prejudices, this is plain distorting of the truth. If you are concerned about this associations, don't shoot the messenger, but the people writing the original texts.
  - jhanschoo 10y ago
    > this is plain distorting of the truth. In the context of business and many other interests, it's expedient to present a more polite face. You're providing a service, not a mirror. In the context of a search engine service, say, a specialized, domain-specific search for a certain profession, such biases often distract and detract from the quality of the service you provide to your customers.

troels 10y ago

"Instead, we suggest that mitigating prejudice should be a separate component of an AI system." Much like I imagine we humans deals with the concept.

deleted 10y ago

[deleted]

nix0n 10y ago

Similarly, algorithms trained to mimic human decisions will mimic human biases. See https://news.ycombinator.com/item?id=11753805 https://news.ycombinator.com/item?id=11753805 (probably others too)

yummyfajitas 10y ago
That article is fundamentally dishonest. The author's own statistical analysis (see their R-script and my comments on the article) cannot reject the null hypothesis that the algorithm is unbiased. I wrote a more detailed critique here: https://www.chrisstucchio.com/blog/2016/propublica_is_lying.html https://www.chrisstucchio.com/blog/2016/propublica_is_lying....

yummyfajitas 10y ago

There was a great comment here discussing the Pathetic fallacy, and how we humans apply it to machine learning systems. (It's now deleted and I don't know why.) Specifically, we treat machine learning systems as anthropomorphic, and assume they will reproduce our biases. https://en.wikipedia.org/wiki/Pathetic_fallacy https://en.wikipedia.org/wiki/Pathetic_fallacy In reality, ML systems will generally correct human biases. And by bias, I really do mean bias in the statistical sense - systematically getting things wrong in a particular direction. Now this article does a great job of explaining how ML systems might understand the meaning of words, and that meaning may contain bias. However, such a system is merely an input into a separate system which actually makes decisions based on those inputs. Extracting meaning from text makes no decisions of it's own. If that later system wants to make accurate decisions, then the best way to do that is to correct for the aforementioned bias, assuming that bias is really bias as opposed to just a correct but undesirable belief about the world [1]. I wrote a blog post a while back that goes into this idea with a bit more math, and which demonstrates some real world "learning" algorithms (mostly linear regression) actually correcting biases: https://www.chrisstucchio.com/blog/2016/alien_intelligences_and_discriminatory_algorithms.html https://www.chrisstucchio.com/blog/2016/alien_intelligences_...

wyager 10y ago
One problem you will run into here is that political use of the word "bias" isn't any sort of statistical claim; it's often just used to mean that something violates vague social expectations about what information is "acceptable" to use when making decisions. ML algorithms don't care; they will make the best possible decision based on the information they have, even if their though process (so to speak) is "biased" in the political sense that it may use gender, race, nationality, etc. to help make optimal decisions.
- yummyfajitas 10y ago
  Yes, unfortunately the term is overloaded. "Bias" can also mean "making correct decisions that I wish were incorrect". That's why I explicitly defined "bias". In the past era, e.g. 1980-2010, it was possible to use vague emotive language to support all kinds of disparate things. As a concrete example that I touch on in my post, and since racism is the loaded undercurrent of this example, we like to pretend that eliminating racial or sexual bias (in the sense of making wrong decisions) will get us proportional representation. Algorithms are bringing us to an age where analytic philosophy is becoming really important. You can tell an algorithm to give you proportional representation, or you can tell it to be racially/sexually unbiased. But the algorithm will reflect reality and reality may not agree with your assumptions; you can't assume that asking for one will give you the other. So now we get into trolley problems: how much meritocracy/equal opportunity will you sacrifice to get proportional representation? Unlike before, this is now a choice you need to explicitly and openly state and acknowledge.

shanacarp 10y ago

Can you measure linguistic shifts with your model?