8 ms·
King – man + woman is queen; but why? (2017)
- CLPadvocate 10mo agofrom Indoeuropean "gwen" over scandinavian "kvinna" = Woman
- shantara 9mo agoThank you. I knew “woman” is “kvinder” in Danish, but I never made the connection with the English word for “queen”.
- memsom 9mo ago"Queen" came from "The king and his queen". There is no common word for Queen in Germanic languages, and for what ever reason Queen became synonymous with royalty. Originally it just mean "the king and his woman", but I don't know when it changed. Certainly we had more than one word for "adult female human" in old English.
- atiedebee 9mo ago> There is no common word for Queen in Germanic languages Correct me if I am misunderstanding what you meant, but Dutch has koningin and German has Königin? They are basically a feminized version of king.
- aitchnyu 9mo agoDid Japanese get a similar word by coincidence?
- pjc50 9mo agohttps://jisho.org/search/queen https://jisho.org/search/queen : which word did you mean? The translator's curse of a language having lots of synonyms, the subtleties of which don't map directly on to English. None of those seem particularly similar to queen/kvinne?
- aitchnyu 9mo agoI mean https://jisho.org/word/%E5%A5%B3 https://jisho.org/word/%E5%A5%B3
- stared 9mo agoThere is a nice "all women are queen, literally", https://www.youtube.com/shorts/kOY-vXuAiBA https://www.youtube.com/shorts/kOY-vXuAiBA And a longer text, https://blog.oup.com/2011/10/wife/ https://blog.oup.com/2011/10/wife/ Also, gynecology has the same roots.
- DonaldFisk 9mo agoAnother cognate is Classical Greek γυνή, whence gynaecology.
- jinushaun 9mo agoArticle is about AI and vector spaces, but queen literally means ”wife/woman” in old English. Disappointed. Thought this would be an article on etymology.
- metalman 9mo agoit is about etymology and how tech companys continiously hijack words, definitions, and whole concepts along with the worlds money, power, and authority, the whole move fast and break stuff thing now running up against people who are objecting,pointedly, increasingly with sticks, and perhaps more tellingly, with genuine befuddled disapointment to find the grasping grinding overreach happening in every last tiny purely intelectual nook and crany.
- amelius 9mo ago[flagged]
- tialaramex 9mo agoUnlike the 21st century US, historically it often didn't go well for kids who became monarch. Henry VI doesn't get murdered early on and his regents do a reasonable job but he did end up dying in the Tower anyway because of the Wars of the Roses which is arguably his fault. Edward V for example was 12 when he became King of England. His uncle sent him to the Tower of London, "for safety" and he never came out, it is generally assumed he was murdered. Edward VI dies before reaching majority, and seems mostly to have been used as a device to control England by older men. Liz I is sometimes portrayed as being a young girl but she's in her mid-20s and has survived Henry VIII's court, so nobody was going to disappear her to the Tower. Trump is of course an old man with only the mind of a toddler. But England never had any of those, and it has long since moved (after killing Charles I) to a constitutional monarchy, separating the figurehead role (for which I think Trump was suited) from the executive role (for which I think it would be hard to imagine a worse candidate).
- actionfromafar 9mo agoSuited like Andrew.
- tialaramex 9mo agoSure, Andrew is just some wealthy creep, Trump seems to be more crooked and stupider but it's not a competition.
- ForceBru 9mo agoYou shoehorned politics into a completely unrelated discussion, consequently making it worse
- zkmon 9mo agoOk, let's say "cash is king". What is queen? Arithmetic is king = royalty + male, while queen = royalty + female But then it makes all these words just arithmetic values without meaning. Even if the words "royalty" and "male" can be sum or difference of some other words and so on - all are just numbers, no meaning at all.
- kleene_op 9mo agoThe meaning is not in the numbers themselves, but in how they each relate to one another. Also those are not mere numbers here, but vectors. Dimensionality and orthogonality is key to define complex relationships.
- Antibabelic 9mo agoThis is my problem with people claiming that LLMs "understand". What we usually call "meaning" in intricately related to an encyclopedic knowledge of the world around us. How does this kind of knowledge not get into the same kind of loop you've just described? It is ultimately founded on our direct experience of the world, on sense data. It is ultimately embodied knowledge.
- viccis 9mo agoVector spaces and bag of words models are not specifically related to LLMs, so I think that's irrelevant to this topic. It's not about "knowledge", just the ability to represent words in such a way that similarities between them take on useful computational characteristics.
- Jorge1o1 9mo agoWell, pretty much all of the LLMs are based on the decode-only version of the Transformer architecture (in fact it’s the T in GPT). And in the Transformer architecture you’re working with embeddings, which are exactly what this article is about, the vector representation of words.
- TeMPOraL 9mo ago
- petesergeant 9mo agoShameless plug of my own article which I think does a much better job of explaining this: https://sgnt.ai/p/embeddings-explainer/ https://sgnt.ai/p/embeddings-explainer/
- stared 9mo agoAuthor here. Thank you for this excavation! I fixed some math rendering - it has suffered after some migration.
- amelius 9mo agoIn the example abs(woman) > abs(man). Any reason why this is the case?
- fruitworks 9mo agoPerhaps femininity is a more specific concept than masculinity. I call this the "Mrs Pac Man Effect". It is more common for people to personify objects (say, a rock or a frog or a random internet user) as male than female. In many languages, the plural for a group of things is male even if it only has one male element. A simple charachter like pac man is male in a universal kiki/bouba sense, but the female equivalent needs a bow (it is a more complex specalization of its male counterpart) Obviously, biologicially male-ness is a specialization of female-ness, because females more resemble unsexed creatures in their ability to reproduce. But in the latent space of the human mind, or in language I think male is closer to "default" and female is a specialzation of maleness. Even the words female or woman are modifications of the word male or man. Perhaps this evolved out of primative social structures. If women occupy a more domestic social role and men a more nomadic one, then you would encounter more men in the outer world. So you would generally associate unknown things in the external world as being masculine, and would associate feminity with specific inter-village inter-family things of your local world
- throw_await 9mo agoThe word female does not, in fact, decompose as fe+male; instead it comes from the same latin root as feminine. The word was influenced over time to more resemble male only because people thought they were related. https://en.wiktionary.org/wiki/female https://en.wiktionary.org/wiki/female
- lblock 9mo agoIsn't the whole linear operations thing not really true and even less so now in embeddings from transformer based models? I remember reading a blog about this but cannot find it anymore. This is the closest thing i could find now: https://mikexcohen.substack.com/p/king-man-woman-queen-is-fake-news https://mikexcohen.substack.com/p/king-man-woman-queen-is-fa...
- mjhay 9mo agoVery much so. It was a little less untrue with older word embedding models* but that kind of semantic linearity never was a thing in practice. Word embedding models try to embed semantically similar words close to each other, but that does not imply linearity at all. *with transformer models, it is pretty much not even wrong.
- wongarsu 9mo agoI just tried it on qwen3-embedding:8b with a little vibe-coded 100 line script that does the obvious linear math and compares the result to the embeddings of a couple of candidate words using cosine similarity, and it did prefer the expected words. Same 22 candidates for both questions king - man + woman ≈ queen (0.8510) Top similarity 0.8510 queen 0.8025 king 0.7674 princess 0.7424 woman 0.7212 queen Elizabeth Berlin - Germany + France ≈ Paris (0.8786) Top similarity 0.8786 Paris 0.8309 Berlin 0.8057 France 0.7824 London Sure, 0.85 is not an exact match so things are not exactly linear, and if I dump an entire dictionary in there it might be worse, but the idea very much works Edit: after running a 100k wordlist through qwen3-embedding:0.6b, the closest matches are: king – man + woman ≈ queen (0.7873) berlin – germany + france ≈ paris (0.9038) london – england + france ≈ paris (0.9137) stronger – strong + weak ≈ weaker (0.8531) stronger – strong + nation ≈ country (0.8047) walking – walk + run ≈ running (0.9098) So clearly throwing a dictionary at it doesn't break it, the closest matches are still the expected ones. The next closest matches got a lot more interesting too, for example the four closest matches for london – england + france are (in order) paris, strasbourg, bordeaux, marseilles
- damnitbuilds 9mo agoactor - man + woman is actress and always will be, no matter how woke you are.
- DonaldFisk 9mo agoIt's telling that king – man + woman = queen is the only example I've ever seen used to explain word2vec. I prefer the old school king(X) :- monarch(X), male(X). queen(X) :- monarch(X), female(X). queen(X) :- wife(Y, X), king(Y). monarch(elizabeth). female(elizabeth). wife(philip, elizabeth). monarch(charles). male(charles). wife(charles, camilla). ?- queen(camilla). true. ?- king(charles). true. ?- king(philip). false. where definitions are human readable rules and words are symbols.
- maxweylandt 9mo agoI've seen in readings (and replicated myself on a set of embeddings derived from google books/news) the capital cities: Berlin - Germany + France = Paris , that sort of thing
- DonaldFisk 9mo agoagain, capital(germany, berlin). capital(france, paris). is clearer. Someone once told me you need humongous vectors to encode nuance, but people are good at things computers are bad at, and vice-versa. I don't want nuance from computers any more than I want instant, precise floating point calculations from people.
- iterateoften 9mo agoI think you are missing the difference between a program derived from training data and logic explicitly created. Go ahead and proceed to continue doing what you are doing for all words in the dictionary and see how the implementation goes.
- DonaldFisk 9mo agoIt depends whether you want your system to handle all of natural language and give answers which are correct most of the time (but it isn't easy to tell when it's wrong), or to handle a limited subset of natural language and either give answers which are demonstrably correct (once it's fully debugged or proven correct), or tells you when it doesn't know the answer. These are two opposing approaches to AI. Rule induction is somewhere in between - you use training data and it outputs (usually probabilistic) human-readable rules.