6 ms·
At first glance this doesn't seem that surprising. We often use "is" in a way which isn't reversible. e.g. "A dog is an animal" -> Makes sense "An animal is a
by steveridout 3y ago
At first glance this doesn't seem that surprising. We often use "is" in a way which isn't reversible. e.g.
"A dog is an animal" -> Makes sense
"An animal is a dog" -> Doesn't make sense
- lordnacho 3y agoYeah isn't this one of those logic things? Perhaps what they mean is NotB -> NotA, which often uses a symbol that maybe is being erased? In any case the abstract seems wrong.
- DebtDeflation 3y agoYes. Modus Ponens vs Affirming The Consequent. If A then B. A. Therefore, B. -> Valid. If A then B. B. Therefore, A. -> Not valid.
- anonzzzies 3y agoYeah, that seems just unclear language and because it's trained on human language, 'is' does not equal 'equals'. Using 'equals' will help.
- eloisant 3y agoThat's the whole problem of LLM: they work only on human language. Even before computers we created formal languages (mathematics, logic equations) precisely because human language is too often ambiguous.
- Spivak 3y agoDon't you lump math in there, math is 99% human language. The symbol pushing you learned in HS is just advanced arithmetic. Math is more like legalese with some very loose additional notation than a formal language.
- lkirkwood 3y agoCan you expand on this? Notation like "=" can be written using language but we define an exact meaning for the operator regardless, unlike language.
- JieJie 3y agoI suppose that begs the question, what if we trained an LLM only on examples of formal languages?
- DonaldFisk 3y agoIn the particular cases being discussed, there's no ambiguity: "is a" means "member of" and "is the" means equals.
- dragonwriter 3y ago> In the particular cases being discussed, there's no ambiguity: "is a" means "member of" and "is the" means equals. Yes, and fitting just those cases would result in a model that handled other cases incorrectly, because idioms inconsistent with that rule exist. (“Jodie is the bomb” has a meaning distinct from the individual words taken separately which is not stating a reflexive equivalency, for instance.)
- TZubiri 3y agoRain is wet. Wet is not rain.
- robjan 3y agoWet is an adjective
- Scarblac 3y agoAdjectives are wet.
- dataflow 3y agoRain is water. Water is not rain.
- dragonwriter 3y agoBirds are dinosaurs. Dinosaurs are not birds. At least not generally. “Birds” and “Dinosaurs” are nouns.
- demondemidi 3y agoDepends on the meaning of the word “is”?
- tmalsburg2 3y agoThis is a useful observation, but it doesn’t explain the particular example given in the article.
- TZubiri 3y agoNot all relations are order independent, so the LLM just assumed none are, prioritizing not being incorrect over being correct.
- V__ 3y agoI would have anticipated that, with a large enough dataset, the latent space would create graph-like relationships. Encoding things many-to-many, one-to-one etc. To my limited understanding this is a surprising find.
- Majromax 3y ago> At first glance this doesn't seem that surprising. We often use "is" in a way which isn't reversible. e.g. They appear to only be testing the 'reliable' cases. There schematic example was fine-tuning the model on "<Fictitious name> is the composer of <fictitious album>", yet having the model be unable to answer "Who composed <fictitious album>"? In this case, English and common sense force symmetry on 'is'. Without further specification, these kinds of prompts imply an exclusive relationship. Additionally, the authors claim that when they tested it, the model didn't even rate the correct answer more probable than random chance. This suggests that the model isn't being clever about logical implications.
- phire 3y agoTo us, it's obvious that "is" in these examples is symmetrical. But LLMs don't have common sense, they have to rely on the training dataset we feed them. It's entirely possible there is nothing wrong with the logical reasoning abilities of LLM architectures and this result is simply an indication the training data doesn't provide enough infomation for LLMs to learn the symmetrical/commutative nature of these "is" relationships. Though, based on the find-the-next-token architecture of LLMs, it seems logical that LLM should need to learn facts in both directions. If it's input set contains <Fictitious name>, it makes sense the tokens for "<fictitious album>" and "composer" will show up with high probability. But there is no reason that having the tokens "composer" and "<fictitious album>" in the input set should increase the probability of the "<fictitious name>" token, because that ordering never occurred in the training data. If true, it would would suggest that LLMs have a massive bias against the very concept of symmetrical logic and commutative operations.
- diffeomorphism 3y agoEnglish only forces that if there is a definite article "the" (unique composer). If it instead said "a" composer, then it is impossible to answer "who composed" completely; you only know one of the composers. Jumping to conclusions like "if A then B" to "A=B" is a very common mistake for humans, bad statistics and propaganda. So I am actually positively surprised that models don't make that mistake.
- wongarsu 3y ago
- robjan 3y agoI think the key words are "a" vs "the" when you use "a" the relationship is one to many, whereas when you use "the" it's one to one. If I say "Charles is the King" then "the King is Charles" also holds true. If I say "Charles is a King" then I can't conclude that the King is Charles.
- beardyw 3y agoSo "dogs are animals", does that work?
- diffeomorphism 3y agoYes, "dogs are the animals (e.g. the only animals on this space station)" is implied to be reversible. Indefinite or missing articles like in your example make no such implication.
- DonaldFisk 3y agoYour examples use the indefinite article, but the first example in the abstract uses the definite article. (The second, after rephrasing, also does.) Contrast "Mars is the fourth planet from the Sun" and "Mars is a planet". With GOFAI (e.g. Cyc, SHRDLU), you'd distinguish between "X is a Y" and "X is the Y" and store them differently, and if you got an incorrect answer you'd have a good idea where to look for your bug. With a LLM, you have a black box with billions of connexion weights and (correct me if I'm wrong) your only recourse is to retrain it on data which distinguishes the two cases, but even that might get lost in the noise, or cause problems somewhere else.
- drt5b7j 3y agoIt depends upon what the meaning of the word "is" is.
- ahartmetz 3y agoIn this case, whether it's an identity-is or a "is a member of the group".
- smusamashah 3y agoThere is a plant which mimics leaves of nearby plants. Try asking GPT-4 which plant it is and it will always give you wrong answers. But if you do give it the name of that plant and ask what it is known for, it will tell you that it can mimic leaves of other plants. This is what their inability to infer A from B is about.