3 ms·
Interesting, definitely need to think about debiasing. Seems like it won't really work straight out of the box since it'd destroy the 1 bit-ness of the vectors.
by maxlam 9y ago
Interesting, definitely need to think about debiasing. Seems like it won't really work straight out of the box since it'd destroy the 1 bit-ness of the vectors.
Though if only a few of the vectors are de-biased then you can still save a lot of space since all the other vectors are still represented using 2 numbers (while the de-biased vectors are represented using the full range of 32 bit numbers).
- rspeer 9y agoThere may be a way to build it into the loss function so that it happens before the quantization, right? (and holy crap, look how fast the HN conservatives are getting to my comment)
- maxlam 9y agoHm not sure, would need to think more about this -- definitely an interesting idea though!
- rspeer 9y agoThanks for being willing to discuss it. And sorry if I'm complicating your task. But any interesting release of NLP data has the potential to affect the way the field progresses, so take it as a compliment that I consider this an interesting release of NLP data. That's why I'm asking you to actively consider the downstream effects of word vectors and find out if you can make them better.
- maxlam 9y agoThanks for the compliments! And thanks also for bringing up the debiasing! It's interesting to see what people care about and any conversation that leads to new ideas is a plus.
- deleted 9y ago[deleted]
- hnuser1234 9y agoLiterally everything is either Left or Right. Anyone who disagrees with you is definitely a sexist conservative. No other possibility. The center is a black hole.