3 ms·

In modern linguistics grammar is used descriptively rather than prescriptively, i.e. it's about describing how speakers actually use language rather than being

In modern linguistics grammar is used descriptively rather than prescriptively, i.e. it's about describing how speakers actually use language rather than being about telling them how to use language 'properly'. If there's gender bias in certain words it might be interesting to point that out but it's not the linguists' job to 'de-bias' the grammar and the underlying model. There's an undeniable gender bias in certain words (men probably are less frequently referred to as 'beautiful' than women while on the other hand 'genius' likely is more often used when referring to men). Glossing over that by smoothing models not only misrepresents how speakers use language but probably doesn't really help the cause either. If the corpus used displays a disproportionate association of certain words with one gender this could just mean that the corpus is insufficient for representing a language in general (as opposed to just a particular register or sociolect), which is a common problem in computational linguistics, not just when it comes to gender biases.

IAmEveryone 9y ago
> it's not the linguists' job to 'de-bias' the grammar and the underlying model. Sure, but this is not a project of linguistic research. It’s a tool that could see real-world usage. Not perpetuating the stereotypes that suffuse the data we feed into such models does seem like a worthy Endeavour, and I find it encouraging to see the creator taking these concerns seriously in this threat. (As opposed to the community-at-large, which quickly send them to the bottom of the threat)
- fvdessen 9y ago
  The issue is that the undesired language constructs one would like to remove are not universal. What is an improvement for one user group will be a regression for another & vice versa.
  - IAmEveryone 9y ago
    I’m a white, able-bodied, mostly heterosexual man, and I reject the theory that (non)discrimination is a zero-sum game.
    - yorwba 9y ago
      Debiasing word vectors is not guaranteed to yield less discriminatory results in every application. In many languages, gender is expressed grammatically and there are different words for e.g. male and female scientists. If you debias the word vectors to remove associations with gender, it becomes impossible to distinguish those two. Now if you translate between two languages with that distinction, the model will have to fix itself to one of the two possibilities, and the translation error will be minimized by always picking the more common form. "First year of all-female Nobel prize winners and machine translation gets it totally wrong!" By debiasing the word vectors, the model has become more biased and less accurate overall. If you create a discriminatory machine learning model, it's unlikely to be because you didn't debias your input representation. More often, you're training it on a task whose real-world statistics are biased, and the model learns to accurately reflect that bias to solve your task. The solution to that is not to modify the input, but rather the output you expect the model to provide.
      - rspeer 9y ago
        Ah yes. This approach to ML bias is popular in some large corporate research groups: - Say a lot of nice things about fairness - Talk about the importance of de-biasing and doing it as well as possible - Show (legitimately) that the farther downstream in your ML process you apply de-biasing, the more sound the results are - Assume that any successful ML model will be used downstream in something else - Therefore, never de-bias anything, except the final output of something where your company benefits from showing fairness: that is, a demonstration at a conference talk about fairness - The time to de-bias real applications is always "later" and everyone can say they are working on doing the right thing People who release data artifacts have the ability to de-bias now instead of later. It's not perfect, but it is good. If you rely on the developer after you to do the de-biasing that you're not doing, you're ensuring that de-biasing won't happen, because they won't do it either.
        yorwba 9y ago
        My point is that the appropriate debiasing is task-specific. You can completely eradicate bias one one task, but as soon as someone builds on top of your results, the bias is going to creep right back in, and likely to be worse due to the information loss in previous layers. If you just hand a bias-conscious developer a bunch of debiased word vectors with the implication that it will make their model less biased, I don't think that's really helpful. Like security, bias in machine learning needs to be addressed on the level of complete systems, it's not something you can leave to some simple input sanitizer.
empath75 9y ago
There’s two purposes for using such a corpus, one is to get an accurate representation of how language is used. Another is to use it as part of a machine human interface, in which case you want to be careful about whether you offend your customers or users or whether you’re even violating the law by biasing certain decisions somehow.
rspeer 9y ago
The corpus alone can't be sufficient. The corpus displays associations that are disproportionate for how we want to use it, because corpora are from the past. But the point of ML is to build something that will be used in the future. [1] There is no objective way to describe this difference between your training set and your test set, because you don't have data from the future. It requires a conscious decision. Enforcing that the future should be like the past is a conscious decision, and it's a lazy and unfortunate one. This is just one instance of a problem that ML is already quite familiar with, and has been since "ML" was called "statistics": observed data produces a biased estimator of the actual distribution. You always have to correct your objectively-measured distributions for what you can't measure objectively. Here I mean "biased" in the mathematical sense; when it comes to human biases encoded in word embeddings, it also produces bias in the ethical sense. But there are many simpler instances of this. For example: a maximum-likelihood language model assigns a probability of 0 to any word it has never seen. Maximum likelihood is, when measured, a better model of the input data than anything else. If you implement this completely objective language model, it would be useless when applied; it would output the impossible probability of 0 for most inputs. Instead you have to "smooth" and "correct" it for the fact that other words exist. [1] https://twitter.com/random_walker/status/975700725807439879 https://twitter.com/random_walker/status/975700725807439879

IAmEveryone 9y ago

> it's not the linguists' job to 'de-bias' the grammar and the underlying model. Sure, but this is not a project of linguistic research. It’s a tool that could see real-world usage. Not perpetuating the stereotypes that suffuse the data we feed into such models does seem like a worthy Endeavour, and I find it encouraging to see the creator taking these concerns seriously in this threat. (As opposed to the community-at-large, which quickly send them to the bottom of the threat)

fvdessen 9y ago
The issue is that the undesired language constructs one would like to remove are not universal. What is an improvement for one user group will be a regression for another & vice versa.
- IAmEveryone 9y ago
  I’m a white, able-bodied, mostly heterosexual man, and I reject the theory that (non)discrimination is a zero-sum game.
  - yorwba 9y ago
    Debiasing word vectors is not guaranteed to yield less discriminatory results in every application. In many languages, gender is expressed grammatically and there are different words for e.g. male and female scientists. If you debias the word vectors to remove associations with gender, it becomes impossible to distinguish those two. Now if you translate between two languages with that distinction, the model will have to fix itself to one of the two possibilities, and the translation error will be minimized by always picking the more common form. "First year of all-female Nobel prize winners and machine translation gets it totally wrong!" By debiasing the word vectors, the model has become more biased and less accurate overall. If you create a discriminatory machine learning model, it's unlikely to be because you didn't debias your input representation. More often, you're training it on a task whose real-world statistics are biased, and the model learns to accurately reflect that bias to solve your task. The solution to that is not to modify the input, but rather the output you expect the model to provide.
    - rspeer 9y ago
      Ah yes. This approach to ML bias is popular in some large corporate research groups: - Say a lot of nice things about fairness - Talk about the importance of de-biasing and doing it as well as possible - Show (legitimately) that the farther downstream in your ML process you apply de-biasing, the more sound the results are - Assume that any successful ML model will be used downstream in something else - Therefore, never de-bias anything, except the final output of something where your company benefits from showing fairness: that is, a demonstration at a conference talk about fairness - The time to de-bias real applications is always "later" and everyone can say they are working on doing the right thing People who release data artifacts have the ability to de-bias now instead of later. It's not perfect, but it is good. If you rely on the developer after you to do the de-biasing that you're not doing, you're ensuring that de-biasing won't happen, because they won't do it either.
      - yorwba 9y ago
        My point is that the appropriate debiasing is task-specific. You can completely eradicate bias one one task, but as soon as someone builds on top of your results, the bias is going to creep right back in, and likely to be worse due to the information loss in previous layers. If you just hand a bias-conscious developer a bunch of debiased word vectors with the implication that it will make their model less biased, I don't think that's really helpful. Like security, bias in machine learning needs to be addressed on the level of complete systems, it's not something you can leave to some simple input sanitizer.

empath75 9y ago

There’s two purposes for using such a corpus, one is to get an accurate representation of how language is used. Another is to use it as part of a machine human interface, in which case you want to be careful about whether you offend your customers or users or whether you’re even violating the law by biasing certain decisions somehow.

rspeer 9y ago

The corpus alone can't be sufficient. The corpus displays associations that are disproportionate for how we want to use it, because corpora are from the past. But the point of ML is to build something that will be used in the future. [1] There is no objective way to describe this difference between your training set and your test set, because you don't have data from the future. It requires a conscious decision. Enforcing that the future should be like the past is a conscious decision, and it's a lazy and unfortunate one. This is just one instance of a problem that ML is already quite familiar with, and has been since "ML" was called "statistics": observed data produces a biased estimator of the actual distribution. You always have to correct your objectively-measured distributions for what you can't measure objectively. Here I mean "biased" in the mathematical sense; when it comes to human biases encoded in word embeddings, it also produces bias in the ethical sense. But there are many simpler instances of this. For example: a maximum-likelihood language model assigns a probability of 0 to any word it has never seen. Maximum likelihood is, when measured, a better model of the input data than anything else. If you implement this completely objective language model, it would be useless when applied; it would output the impossible probability of 0 for most inputs. Instead you have to "smooth" and "correct" it for the fact that other words exist. [1] https://twitter.com/random_walker/status/975700725807439879 https://twitter.com/random_walker/status/975700725807439879