4 ms·
I think it very unfair to the original set of word2vec papers to be talking about 'academic dishonesty'. This is a case of a user that has little to no experien
by utopcell 7y ago
I think it very unfair to the original set of word2vec papers to be talking about 'academic dishonesty'. This is a case of a user that has little to no experience with neural networks. There are a ton of articles describing the need for random initialization [1][2]. In fact, if one spends a few seconds thinking about it, the need is evident. Without it, the NN cannot perform symmetry breaking: If inputs are set to zero, all neurons will perform the same calculations, rendering the network useless.
[1] google: "neural networks vector initialization"
[2] http://deeplearning.ai/ai-notes/initialization/ http://deeplearning.ai/ai-notes/initialization/
- bollu 7y agoThat's hardly the point of the article --- the actual paper does not describe the use of two separate vectors for each word. The initialization was an interesting tidbit.
- exgrv 7y agoExcept it does? After Equation 2: "v_w and v'_w are the input and output vector representations of w."
- utopcell 7y agoOf course it does. For word w they are v_w and v'_w on Eq.4 for the case of SGNS [1]. [1] https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf https://papers.nips.cc/paper/5021-distributed-representation...
- andreasvc 7y agobollu, multiple people have shown that your claim that the paper doesn't match the code is flat out wrong. I think at this point you should issue a retraction of your wildly inappropriate suggestion of academic dishonesty.