4 ms·
Cool. I'm going to be trying this out and comparing to ConceptNet Numberbatch [1] (the current state-of-the-art multilingual vectors). I'd like to see how they
by rspeer 10y ago
Cool. I'm going to be trying this out and comparing to ConceptNet Numberbatch [1] (the current state-of-the-art multilingual vectors). I'd like to see how they compare on existing evaluations of word similarity.
EDIT: never mind the comparison, I just got the part where the monolingual performance should be the same as fastText's. Which I don't think is a great thing. The precomputed fastText vectors aren't very good in most languages because most Wikipedias are small. Even the Japanese fastText vectors perform only slightly better than chance on Sakaizawa's evaluation of Japanese word similarity [2].
I would expect that, having data from both Wikipedia and Google Translate, you should be able to make a system that's monolingually much better than fastText on Wikipedia alone. That's what I was hoping to compare to. Don't limit yourself to the performance of fastText's data.
I also observe that the data files could be made much, much smaller. Every value is written out as a double-precision number in decimal. Most of these digits convey no information, and they're effectively random, limiting the effectiveness of compression. These could all be rounded to about 4 digits after the decimal point with no loss of performance.
[1] https://github.com/commonsense/conceptnet-numberbatch https://github.com/commonsense/conceptnet-numberbatch
[2] https://github.com/tmu-nlp/JapaneseWordSimilarityDataset https://github.com/tmu-nlp/JapaneseWordSimilarityDataset
- nl 10y agoAre the Numberbatch vectors aligned in the same vector space? That seems to be the primary innovation here, and it's a pretty important one.
- rspeer 10y agoThey are. There was a SemEval competition about having vectors that are both accurate for 5 individual languages and aligned between all the pairs of languages [1], where Numberbatch was the first-place system [2]. The fastText_multilingual system does have something going on, that it can discover multilingually aligned vectors after the fact for a system that was previously separate for each language, and leaves it "the same system" in a sense. But incorporating the ConceptNet knowledge graph also gives you a multilingually aligned system. [1] http://alt.qcri.org/semeval2017/task2/ http://alt.qcri.org/semeval2017/task2/ [2] https://blog.conceptnet.io/2017/03/02/how-luminoso-made-conceptnet-into-the-best-word-vectors-and-won-at-semeval/ https://blog.conceptnet.io/2017/03/02/how-luminoso-made-conc...
- sls56 10y agoCool rspeer, we will check NumberBatch out! Yes, as you say, our goal here is to show you can get something for nothing from pre-trained embeddings (I can learn the 78 matrices on my macbook in about ten minutes...). The alignment procedure is completely agnostic to what word vectors you use. It only requires that cosine similarity is a good distance metric. We used fastText because there were lots of open-sourced vectors available...