4 ms·
To save a few clicks, here's the paper that describes the fix and gives some comparisons with the supposedly broken implementation. https://arxiv.org/pdf/2012.
by steve_g 6y ago
To save a few clicks, here's the paper that describes the fix and gives some comparisons with the supposedly broken implementation.
https://arxiv.org/pdf/2012.15332.pdf https://arxiv.org/pdf/2012.15332.pdf
- gojomo 6y agoWhile I'm unsure of this paper/implementation's main claims without a closer reading, the Appendix D 'alias method for negative sampling' looks like it might be a nice standalone performance improvement to Gensim (& others') negative-sampling code.
- utopcell 6y agoIndeed it is a significant performance boost. I believe I was the first to suggest using the alias method for w2v. I had extended it to work in a distributed setting a few years back [1] which allowed me to scale the problem to a dataset of 1 trillion words on a lexicon of 1.4BN words (roughly, the top 40 billion web pages of Yahoo's search engine). [1] https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14956 https://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14956