4 ms·
Noticed in the code no Huffman tree. Then FN2 in the paper: "In this work, we always use the negative sampling formulations of Word2vec objectives which are con
by piker 6y ago
Noticed in the code no Huffman tree. Then FN2 in the paper: "In this work, we always use the negative sampling formulations of Word2vec objectives which are consis-tently more efficient and effective than the hierarchical softmax formulations." Is that consensus?
- gojomo 6y agoIt's the default most places. Especially with larger training corpora or vocabularies, the negative-sampling tends perform better - & I don't recall notable situations where the HS mode is better. (I recall seeing some hints in early word2vec code of an HS-based vocabulary that wasn't based on mere word-frequency, but some earlier or perhaps iterated semantic-clustering steps, that I think managed to give similar words shared codes. But I've not seen more on that recently.)