3 ms·
The original paper included source, and that has their test data and results -- it gets ~77% accuracy on about 20k example word analogies (with 99.7% coverage),
by kevindamm 3y ago
The original paper included source, and that has their test data and results -- it gets ~77% accuracy on about 20k example word analogies (with 99.7% coverage), and 78% accuracy with phrases with 77% coverage (and a much smaller data set, 3,218 examples). You can see the test sets here:
https://github.com/tmikolov/word2vec/blob/master/questions-words.txt https://github.com/tmikolov/word2vec/blob/master/questions-w...
https://github.com/tmikolov/word2vec/blob/master/questions-phrases.txt https://github.com/tmikolov/word2vec/blob/master/questions-p...
and you could see how much better LLMs do on the same 20k examples.