There are tons of ways to evaluate word vector quality! Word analogy tasks, word similarity tasks, contextual prediction tasks, etc. This link contains a bunch

There are tons of ways to evaluate word vector quality! Word analogy tasks, word similarity tasks, contextual prediction tasks, etc. This link contains a bunch of relevant evaluation datasets and benchmarks obtained using word2vec, GloVe, etc. You can evaluate your RNN-learned vectors and compare them to a traditionally trained word2vec-trained vectors. Link here: http://www.bigdatalab.ac.cn/benchmark/bm/Domain?domain=Word%20Representation http://www.bigdatalab.ac.cn/benchmark/bm/Domain?domain=Word%... For more background on evaluating word vectors check out these pretty great lecture notes from Socher's NLP class: http://cs224d.stanford.edu/lecture_notes/LectureNotes2.pdf http://cs224d.stanford.edu/lecture_notes/LectureNotes2.pdf Also, here's the original papers from a few years ago that introduced many of these datasets and evaluation standards: https://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf https://papers.nips.cc/paper/5021-distributed-representation... http://www.cs.cmu.edu/~mfaruqui/papers/acl14-vecdemo.pdf http://www.cs.cmu.edu/~mfaruqui/papers/acl14-vecdemo.pdf