3 ms·
I ran one experiment with an Embedding layer (second one in the Deep Learning Is No Good Here section), which I assumed to be analagous to word2vec. I hoped it
by andreyk 11y ago
I ran one experiment with an Embedding layer (second one in the Deep Learning Is No Good Here section), which I assumed to be analagous to word2vec. I hoped it would help as well so was rather dissapointed it did not seem to be so. Perhaps I should have tried a few more configurations, though.
- nicklo 11y agoAn Embedding layer by itself will try to learn word vectors from the data you train it on (error gradient will propagate back to embedding layer, update the vector weights). Word2vec and word vectors are only really useful with tons of training data to learn good embeddings. I believe OP is referring to using Google's pre-trained word-vectors (trained on massive amounts of text [100 Billion words]). Can be found here: https://code.google.com/p/word2vec/ https://code.google.com/p/word2vec/ This is pretty straightforward to implement in keras, you just need to supply pre-trained word-vectors weights to your embedding layer. Embedding(vocab_size, 300, weights = [word2vec_weights]) Where 'word2vec_weights' is a numpy matrix with shape (vocab_size, 300).