3 ms·
Yes. There is some nifty work on trying to structure an actual grammar based on the geometry of the embedding space, but "look at this cool thing we found" is
by fiatmoney 11y ago
Yes.
There is some nifty work on trying to structure an actual grammar based on the geometry of the embedding space, but "look at this cool thing we found" is a totally worthwhile thing to publish.
(Embeddings themselves are useful for a whole lot more than that, though.)
- SilasX 11y agoIf these are the most interesting results, then it's not. Most of the work seems to be in the (human provided) search for happy combinations like these. Are they typical? Is the whole space like this? Is there a consistent metric for deciding that these are in fact clever and it's not just the human doing the work in making it make sense in a funny way?
- sp332 11y agoAs pointed out elsewhere, the results are pretty accurate, not nearly perfect but enough to open up new applications. http://arxiv.org/pdf/1301.3781.pdf http://arxiv.org/pdf/1301.3781.pdf Word2vec is exciting because it has a useable amount of accuracy for a relatively small amount of cleverness on the part of the human. You can just throw a huge corpus at it and it performs reasonably well.
- SilasX 11y agoWhat does "accurate" mean in this context? If there's a standard meaning of it, I'd be glad to judge it by that. But the metric the author offered, as their defining results, was some cute things with Putin and Obama. So yeah, if the results are accurate relative to some standard metric, I wasn't disputing that. I was questioning the reliability of the results the author presented, about cute Obama/Putin connections.
- sp332 11y agoIt's not just "cute", that's the whole point. Somewhat surprisingly, it was found that similarity of word representations goes beyond simple syntactic regularities. Using a word offset technique where simple algebraic operations are performed on the word vectors, it was shown for example that vector(”King”) - vector(”Man”) + vector(”Woman”) results in a vector that is closest to the vector representation of the word Queen. In this paper, we try to maximize accuracy of these vector operations by developing new model architectures that preserve the linear regularities among words. We design a new comprehensive test set for measuring both syntactic and semantic regularities, and show that many such regularities can be learned with high accuracy.
- SilasX 11y agoAnd if the author highlighted those as "the results", I would be impressed. As it stands, they were citing more "moonshot" kind of results like the ability to get abstract concept composition right, which is really hard to judge, because you don't know what set to sample over, or whether the equations are really insightful vs "I guess that might make sense...". I'm much more confused over how the author characterized the results ("look at this Animal/Human/Ethics thing!"). Again, if there are standard metrics by which it did well, great! But why isn't he citing them rather than these more abstract, hard-to-judge victories?
- jmalicki 11y agoAs stated above in the thread, it has a 50% accuracy rate on an analogy dataset.