6 ms·
> king-man+woman=queen Is the famous example everyone uses when talking about word vectors, but is it actually just very cherry picked? I.e. are there a great
by montebicyclelo 1y ago
> king-man+woman=queen
Is the famous example everyone uses when talking about word vectors, but is it actually just very cherry picked?
I.e. are there a great number of other "meaningful" examples like this, or actually the majority of the time you end up with some kind of vaguely tangentially related word when adding and subtracting word vectors.
(Which seems to be what this tool is helping to illustrate, having briefly played with it, and looked at the other comments here.)
(Btw, not saying wordvecs / embeddings aren't extremely useful, just talking about this simplistic arithmetic)
- raddan 1y ago> is it actually just very cherry picked? 100%
- gregschlom 1y agoAlso, as I just learned the other day, the result was never equal, just close to "queen" in the vector space.
- charcircuit 1y agoAnd queen isn't even the closest.
- mcswell 1y agoWhat is the closest?
- deleted 1y ago[deleted]
- charcircuit 1y agoUsually king is.
- chis 1y agoI mean they are floating point vectors so
- Retr0id 1y agoI think it's slightly uncommon for the vectors to "line up" just right, but here are a few I tried: actor - man + woman = actress garden + person = gardener rat - sewer + tree = squirrel toe - leg + arm = digit
- groby_b 1y agoI think it's worth keeping in mind that word2vec was specifically trained on semantic similarity. Most embedding APIs don't really give a lick about the semantic space And, worse, most latent spaces are decidedly non-linear. And so arithmetic loses a lot of its meaning. (IIRC word2vec mostly avoided nonlinearity except for the loss function). Yes, the distance metric sort-of survives, but addition/multiplication are meaningless. (This is also the reason choosing your embedding model is a hard-to-reverse technical decision - you can't just transform existing embeddings into a different latent space. A change means "reembed all")
- jbjbjbjb 1y agoWell when it works out it is quite satisfying India - Asia + Europe = Italy Japan - Asia + Europe = Netherlands China - Asia + Europe = Soviet-Union Russia - Asia + Europe = European Russia calculation + machine = computer
- trhway 1y agodemocracy - vote = progressivism I'll have to mediate on that.
- blipvert 1y agoperson + man + woman + camera + television = user
- kgeist 1y agoInteresting: Russia - Europe = Putin Ukraine + Putin = Russia Putin - Stalin = Bush Stalin - purge = Lenin That means Bush = Ukraine+Putin-Europe-Lenin-purge. However, the site gives Bush -4%, second best option (best is -2%, "fleet ballistic missile submarine", not sure what negative numbers mean).
- nxa 1y agoMy interpretation of negative numbers is that no "synonym" was found (no vector pointing in the same direction), and that the closest expression on record is something with an opposite meaning (pointing in reverse direction), so I'd say that's an antonym.
- bee_rider 1y agoHmm, well I got cherry - picker = blackwood if that helps.
- loganmhb 1y agoI once saw an explanation which I can no longer find that what's really happening here is also partly "man" and "woman" are very similar vectors which nearly cancel each other out, and "king" is excluded from the result set to avoid returning identities, leaving "queen" as the closest next result. That's why you have to subtract and then add, and just doing single operations doesn't work very well. There's some semantic information preserved that might nudge it in the right direction but not as much as the naive algebra suggests, and you can't really add up a bunch of these high-dimensional vectors in a sensible way. E.g. in this calculator "man - king + princess = woman", which doesn't make much sense. "airplane - engine", which has a potential sensible answer of "glider", instead "= Czechoslovakia". Go figure.