4 ms·

Evaluation of https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki.en.zip https://s3-us-west-1.amazonaws.com/fasttext-vectors/wiki.en.... (unzips wiki.en.ve

• atrudeau 10y ago
  Google Analogy Task results ( detailed results at http://pastebin.com/PF96nMfX http://pastebin.com/PF96nMfX ): Semantic accuracy: 63.84 % Syntactic accuracy: 67.00 % Here performance is not great (great would be >80% on semantic and >70% on syntactic). As this task requires nearest neighbor lookups, performance is impacted by vocabulary size. Since models trained using Wikipedia alone usually limit vocabulary to something ~300k words, we can try that to get scores which are comparable to those posted by the GloVe [1] and LexVec [2] papers by only using the first 300k words in the pre-trained vectors, giving the following results: Semantic accuracy: 77.75 % Syntactic accuracy: 72.55 % Impressive stuff! [1] http://nlp.stanford.edu/pubs/glove.pdf http://nlp.stanford.edu/pubs/glove.pdf - https://github.com/stanfordnlp/GloVe https://github.com/stanfordnlp/GloVe [2] https://arxiv.org/pdf/1606.01283v1 https://arxiv.org/pdf/1606.01283v1 - https://github.com/alexandres/lexvec https://github.com/alexandres/lexvec