6 ms·

It's interesting and slightly uncomfortable that the illustration for similar words uses the word "man" as an example, given the gender biases that result from

by rspeer 9y ago

It's interesting and slightly uncomfortable that the illustration for similar words uses the word "man" as an example, given the gender biases that result from learning word vectors solely from word distributions. To explain, although I’m sure the author himself is familiar with the issue: For any word that is disproportionately associated with one gender in the corpus, the model will learn that gender difference as part of the representation of the word, pretty much baking it into all applications. [1] It's all fun and games when this helps you find the difference between "king" and "queen", but it becomes a problem when the same difference appears between "genius" and "beautiful". I haven't evaluated these vectors for built-in biases, but I assume they would have similar problems to the pre-computed word2vec and GloVe embeddings. (If they don't -- if quantization is a natural way to counteract bias -- then that's an awesome result! But an unlikely one.) To the author: I don’t mean this to sound like an accusation that you haven’t done this yet; I know that short papers can’t tell two stories at the same time. But the next step is pretty clear, right? How do these vectors measure on the Word Embedding Association Test for implicit bias? Does Bolukbasi’s de-biasing method, or an analogue of it, work on quantized embeddings? [1] Bolukbasi et al., "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings." https://arxiv.org/abs/1607.06520 https://arxiv.org/abs/1607.06520

maxlam 9y ago
Interesting, definitely need to think about debiasing. Seems like it won't really work straight out of the box since it'd destroy the 1 bit-ness of the vectors. Though if only a few of the vectors are de-biased then you can still save a lot of space since all the other vectors are still represented using 2 numbers (while the de-biased vectors are represented using the full range of 32 bit numbers).
- rspeer 9y ago
  There may be a way to build it into the loss function so that it happens before the quantization, right? (and holy crap, look how fast the HN conservatives are getting to my comment)
  - maxlam 9y ago
    Hm not sure, would need to think more about this -- definitely an interesting idea though!
    - rspeer 9y ago
      Thanks for being willing to discuss it. And sorry if I'm complicating your task. But any interesting release of NLP data has the potential to affect the way the field progresses, so take it as a compliment that I consider this an interesting release of NLP data. That's why I'm asking you to actively consider the downstream effects of word vectors and find out if you can make them better.
      - maxlam 9y ago
        Thanks for the compliments! And thanks also for bringing up the debiasing! It's interesting to see what people care about and any conversation that leads to new ideas is a plus.
      - deleted 9y ago
        [deleted]
  - hnuser1234 9y ago
    Literally everything is either Left or Right. Anyone who disagrees with you is definitely a sexist conservative. No other possibility. The center is a black hole.
hnuser1234 9y ago
One of the closest related words is lady, and then effeminate, so it's quite unclear exactly what there is to be upset about? In fact it appears it's saying men are related to being effeminate, which should get you excited.
- strawcomb 9y ago
  What a great quantitative evaluation of bias you just carried out.
  - hnuser1234 9y ago
    Alright let me try to be real. After reading http://www.sciencemag.org/news/2017/04/even-artificial-intelligence-can-acquire-biases-against-race-and-gender http://www.sciencemag.org/news/2017/04/even-artificial-intel... and https://en.wikipedia.org/wiki/Implicit-association_test https://en.wikipedia.org/wiki/Implicit-association_test (in that order thank you), it looks like the implicit association test is a good way to magnify small differences (I'm thinking floating point error) in how much you associate one of a few words with a certain other. So, presumably, quantification would really be a natural way to eliminate those types of subtle differences.
    - rspeer 9y ago
      I'm glad we could at least get you to read it, but you're still not caught up. The effect is many orders of magnitude larger than floating point error. The notion that implicit bias could be corrected by quantization (not "quantification") is an interesting hypothesis, with a low prior probability, which could be tested by experiment and easily published if it is true.
fvdessen 9y ago
Personally I am rather uncomfortable with the idea of 'correcting' the learning process to have the machine output what we think it should instead of what it objectively learns from the dataset. It is trading a bias against an ideal world with a bias against reality.
- rspeer 9y ago
  Where did you get the idea that data, of all things, is objective? Data is shit! Data is the noxious raw material that we have to process with great difficulty into something useful! If you are familiar at all with machine learning, you should recognize that human decisions affect every step of the process, especially the part where the data is produced and collected. It is not an oracle of objective truth. And let me quote Arvind Narayanan for why you are not going to get the right answer in your search for objectivity: "Training data is from the past and test data is from the future. We use ML because we want to learn from the past, not reproduce it." [1] [1] https://twitter.com/random_walker/status/975700725807439879 https://twitter.com/random_walker/status/975700725807439879
  - fvdessen 9y ago
    IIRC, if you remove the 'uh' and 'hmm' from audio samples to train for speech recognition, you get lesser performance than if you leave them in. So it is not that easy to differentiate a-priori between actual data and noise. If you look at the history of NLP we went from 'sentences are instances of a well defined grammar' to 'sentences are a bunch of statistically related tokens'. Embracing the mess is how we got better results ! And as for de-biasing gender terms, It makes the assumption that the gender differences in language only have nefarious purposes, but without an actual proof that this is the case, you may very well throw the baby with the bathwater. A good example of this is when the french government mandated that Resumes must be anonymous and not mention sex, gender, etc. It actually resulted in worse outcome for people from lower socio-economic background. It turned out that on average people were more forgiving of bad Resumes if it came from people who could be expected to be disadvantaged.
    - 9y ago
BjoernKW 9y ago
In modern linguistics grammar is used descriptively rather than prescriptively, i.e. it's about describing how speakers actually use language rather than being about telling them how to use language 'properly'. If there's gender bias in certain words it might be interesting to point that out but it's not the linguists' job to 'de-bias' the grammar and the underlying model. There's an undeniable gender bias in certain words (men probably are less frequently referred to as 'beautiful' than women while on the other hand 'genius' likely is more often used when referring to men). Glossing over that by smoothing models not only misrepresents how speakers use language but probably doesn't really help the cause either. If the corpus used displays a disproportionate association of certain words with one gender this could just mean that the corpus is insufficient for representing a language in general (as opposed to just a particular register or sociolect), which is a common problem in computational linguistics, not just when it comes to gender biases.
- IAmEveryone 9y ago
  > it's not the linguists' job to 'de-bias' the grammar and the underlying model. Sure, but this is not a project of linguistic research. It’s a tool that could see real-world usage. Not perpetuating the stereotypes that suffuse the data we feed into such models does seem like a worthy Endeavour, and I find it encouraging to see the creator taking these concerns seriously in this threat. (As opposed to the community-at-large, which quickly send them to the bottom of the threat)
  - fvdessen 9y ago
    The issue is that the undesired language constructs one would like to remove are not universal. What is an improvement for one user group will be a regression for another & vice versa.
    - IAmEveryone 9y ago
      I’m a white, able-bodied, mostly heterosexual man, and I reject the theory that (non)discrimination is a zero-sum game.

by rspeer 9y ago

maxlam 9y ago

Interesting, definitely need to think about debiasing. Seems like it won't really work straight out of the box since it'd destroy the 1 bit-ness of the vectors. Though if only a few of the vectors are de-biased then you can still save a lot of space since all the other vectors are still represented using 2 numbers (while the de-biased vectors are represented using the full range of 32 bit numbers).

rspeer 9y ago
There may be a way to build it into the loss function so that it happens before the quantization, right? (and holy crap, look how fast the HN conservatives are getting to my comment)
- maxlam 9y ago
  Hm not sure, would need to think more about this -- definitely an interesting idea though!
  - rspeer 9y ago
    Thanks for being willing to discuss it. And sorry if I'm complicating your task. But any interesting release of NLP data has the potential to affect the way the field progresses, so take it as a compliment that I consider this an interesting release of NLP data. That's why I'm asking you to actively consider the downstream effects of word vectors and find out if you can make them better.
    - maxlam 9y ago
      Thanks for the compliments! And thanks also for bringing up the debiasing! It's interesting to see what people care about and any conversation that leads to new ideas is a plus.
    - deleted 9y ago
      [deleted]
- hnuser1234 9y ago
  Literally everything is either Left or Right. Anyone who disagrees with you is definitely a sexist conservative. No other possibility. The center is a black hole.

hnuser1234 9y ago

One of the closest related words is lady, and then effeminate, so it's quite unclear exactly what there is to be upset about? In fact it appears it's saying men are related to being effeminate, which should get you excited.

strawcomb 9y ago
What a great quantitative evaluation of bias you just carried out.
- hnuser1234 9y ago
  Alright let me try to be real. After reading http://www.sciencemag.org/news/2017/04/even-artificial-intelligence-can-acquire-biases-against-race-and-gender http://www.sciencemag.org/news/2017/04/even-artificial-intel... and https://en.wikipedia.org/wiki/Implicit-association_test https://en.wikipedia.org/wiki/Implicit-association_test (in that order thank you), it looks like the implicit association test is a good way to magnify small differences (I'm thinking floating point error) in how much you associate one of a few words with a certain other. So, presumably, quantification would really be a natural way to eliminate those types of subtle differences.
  - rspeer 9y ago
    I'm glad we could at least get you to read it, but you're still not caught up. The effect is many orders of magnitude larger than floating point error. The notion that implicit bias could be corrected by quantization (not "quantification") is an interesting hypothesis, with a low prior probability, which could be tested by experiment and easily published if it is true.

fvdessen 9y ago

Personally I am rather uncomfortable with the idea of 'correcting' the learning process to have the machine output what we think it should instead of what it objectively learns from the dataset. It is trading a bias against an ideal world with a bias against reality.

rspeer 9y ago
Where did you get the idea that data, of all things, is objective? Data is shit! Data is the noxious raw material that we have to process with great difficulty into something useful! If you are familiar at all with machine learning, you should recognize that human decisions affect every step of the process, especially the part where the data is produced and collected. It is not an oracle of objective truth. And let me quote Arvind Narayanan for why you are not going to get the right answer in your search for objectivity: "Training data is from the past and test data is from the future. We use ML because we want to learn from the past, not reproduce it." [1] [1] https://twitter.com/random_walker/status/975700725807439879 https://twitter.com/random_walker/status/975700725807439879
- fvdessen 9y ago
  IIRC, if you remove the 'uh' and 'hmm' from audio samples to train for speech recognition, you get lesser performance than if you leave them in. So it is not that easy to differentiate a-priori between actual data and noise. If you look at the history of NLP we went from 'sentences are instances of a well defined grammar' to 'sentences are a bunch of statistically related tokens'. Embracing the mess is how we got better results ! And as for de-biasing gender terms, It makes the assumption that the gender differences in language only have nefarious purposes, but without an actual proof that this is the case, you may very well throw the baby with the bathwater. A good example of this is when the french government mandated that Resumes must be anonymous and not mention sex, gender, etc. It actually resulted in worse outcome for people from lower socio-economic background. It turned out that on average people were more forgiving of bad Resumes if it came from people who could be expected to be disadvantaged.
  - 9y ago

BjoernKW 9y ago

In modern linguistics grammar is used descriptively rather than prescriptively, i.e. it's about describing how speakers actually use language rather than being about telling them how to use language 'properly'. If there's gender bias in certain words it might be interesting to point that out but it's not the linguists' job to 'de-bias' the grammar and the underlying model. There's an undeniable gender bias in certain words (men probably are less frequently referred to as 'beautiful' than women while on the other hand 'genius' likely is more often used when referring to men). Glossing over that by smoothing models not only misrepresents how speakers use language but probably doesn't really help the cause either. If the corpus used displays a disproportionate association of certain words with one gender this could just mean that the corpus is insufficient for representing a language in general (as opposed to just a particular register or sociolect), which is a common problem in computational linguistics, not just when it comes to gender biases.

IAmEveryone 9y ago
> it's not the linguists' job to 'de-bias' the grammar and the underlying model. Sure, but this is not a project of linguistic research. It’s a tool that could see real-world usage. Not perpetuating the stereotypes that suffuse the data we feed into such models does seem like a worthy Endeavour, and I find it encouraging to see the creator taking these concerns seriously in this threat. (As opposed to the community-at-large, which quickly send them to the bottom of the threat)
- fvdessen 9y ago
  The issue is that the undesired language constructs one would like to remove are not universal. What is an improvement for one user group will be a regression for another & vice versa.
  - IAmEveryone 9y ago
    I’m a white, able-bodied, mostly heterosexual man, and I reject the theory that (non)discrimination is a zero-sum game.