5 ms·
An Embedding layer by itself will try to learn word vectors from the data you train it on (error gradient will propagate back to embedding layer, update the vec
by nicklo 11y ago
An Embedding layer by itself will try to learn word vectors from the data you train it on (error gradient will propagate back to embedding layer, update the vector weights). Word2vec and word vectors are only really useful with tons of training data to learn good embeddings. I believe OP is referring to using Google's pre-trained word-vectors (trained on massive amounts of text [100 Billion words]). Can be found here: https://code.google.com/p/word2vec/ https://code.google.com/p/word2vec/
This is pretty straightforward to implement in keras, you just need to supply pre-trained word-vectors weights to your embedding layer.
Embedding(vocab_size, 300, weights = [word2vec_weights])
Where 'word2vec_weights' is a numpy matrix with shape (vocab_size, 300).