9 ms·
Language models still struggle with the concept of negation
- chewxy 4y agoThe stop words bit is a bit wtf. Someone didn't do their homework
- hhghjn 4y agoI stopped reading after that. Some older NLP flows would eliminate stopwords, but I doubt any would eliminate "not". In any case, modern (i.e., last 5 years or so) LLMs don't
- GolDDranks 4y agoYep, that bothered me too. They are called stop words because, but only if, you filter them out. If you don't, they aren't stop words. There is no inherent "stop-ness" there.
- drcode 4y agoI just typed the examples into chatgpt4 and it got all of them right
- optimalsolver 4y agoWill AI skeptics ever get tired of being btfo by stochastic parrots? Gary Marcus's Twitter timeline says no.
- williamcotton 4y agoWhat animals don’t have paws or lay eggs, but have wings? Work it out, step by step. ChatGPT (GPT-4): Sure, let's go through this step by step: First, you asked for animals that don't have paws. This would exclude mammals such as dogs, cats, bears, and so on, as these species have paws. Second, you specified animals that don't lay eggs. This eliminates a wide range of animals including all birds, most reptiles, and some fish and insects, as they all lay eggs. Finally, you asked for animals that have wings. Wings are typically found in birds, bats, and insects. Given these three criteria, we can see that we're in a bit of a conundrum. Most winged creatures either have paws (like bats) or lay eggs (like birds and insects). However, if we consider the term "wings" more broadly, there are some species of bats that give live birth (don't lay eggs) and technically don't have "paws" but rather winged limbs adapted for flight. So, based on the constraints of your question, the best fit would be certain species of bats, provided we interpret "wings" in a broad sense and "paws" in a strict sense.
- jostmey 4y agoFrom what I’ve seen, there’s a huge difference in chat GTP 4 from other LLM
- jiggawatts 4y agoInteresting that it contradicted itself by saying bats have paws and don’t have paws, but arrived at the correct conclusion anyway.
- hammyhavoc 4y ago"A broken clock is right twice per day" as the saying goes.
- williamcotton 4y agoThat aphorism doesn’t apply here. What is meant by that saying is, “well the first time we ran the machine it worked just fine, so we got all excited, but then it consistently failed after that”, with the lesson being, increase your sample size before you pass judgement. LLMs almost suffer from the opposite. Because they are correct so often the danger is in assuming the response is always correct. Maybe, “even a Swiss made watch needs to be checked against the town bells”?
- deleted 4y ago[deleted]
- usaar333 4y agoYa, it doesn't really struggle with the examples in this article to the point it feels like it invalidates the article.
- hn_throwaway_99 4y agoWhile the answer landed on bats, ChatGPT kind of trips itself up in its explanations: 1. "Most winged creatures either have paws (like bats) or lay eggs (like birds and insects)" - but bats don't have paws. 2. "However, if we consider the term "wings" more broadly" - why do we need to consider the term "wings" more broadly? Nobody is arguing there is some definition of "wing" that would exclude bats. Bat wings evolved from largely the same structures that bird wings did. 3. "there are some species of bats that give live birth" - all species of bats give live birth - stating it like this makes it sound like there are some species of bats that lay eggs or something. 4. "...provided we interpret "wings" in a broad sense and "paws" in a strict sense." Again, no "broad" or "strict" senses are needed. Bats have wings and no paws. To be clear, I still think ChatGPT is amazing, but these kinds of things highlight some of the differences between LLMs and human cognition.
- jamal-kumar 4y agoI find I have to state negations on what I want very emphatically in order for it to pick it up, like I'm carefully explaining to someone how to do something for the very first time in a way where they don't screw it up. I'll use stuff like MAKE SURE YOU DO THIS AND NOT THAT as a prompt, in all caps, and it seems to see that as emphasis. I might also even have to repeat them in order for it to get things right but generally gpt-4 seems to pick up on these cues. It definitely feels like a sort of point where it could use some improvement because if I'm not extremely explicit it will give me some issues. As an example: "Can you please recommend me 10 books about witches set in Europe that take place no later than the Napoleonic wars" wasn't enough of an exclusion to work, but "Hi! Can you please recommend me 10 books about witches set in Europe that take place no later than the Napoleonic wars? Please exclude any books that don't take place in Europe, and also exclude any books which take place after The Napoleonic Wars (1803–1815). Thanks, ChatGPT!" was exactly enough to return what we were looking for.
- gregsadetsky 4y agoThis reminded me of https://github.com/elsamuko/Shirt-without-Stripes https://github.com/elsamuko/Shirt-without-Stripes Whose days may be soon over as gpt4 gets this overall right..? As noted by many folks here
- IanCal 4y agoGpt4 + klarna plugin does that just fine. Gpt4 decided to search for plain shirts.
- gregsadetsky 4y agoJust tried GPT4 + Klarna and got 4 suggestions that were all along the lines of "adidas Women's Adicolor Classics 3-Stripes T-shirt - Black" ... :-) I'll open an issue...
- deleted 4y ago[deleted]
- bitL 4y agoCan't this be solved by adversarial training? Just add as many confusing negotiations as possible with the correct outcome sentence/label during fine-tuning and see an LLM to finally get it working?
- akavi 4y agoDoes this increase anyone else's sense that LLMs are actually modeling something similar to the way humans think? It certainly does for me. I find that negations/conjugations of negations are surprisingly difficult for to parse. Beyond two negations at most, I can feel myself having to switch into "code" mode, where I'm explicitly casing out the booleans, to understand them. Obviously the examples given in the article are simpler than double negations, but it feels very consistent with the idea that "negations are an inherently hard part of language processing".
- jprete 4y agoThis specific example doesn’t, but I think there’s a fair amount of brain processing similar to what LLMs do. I just don’t think it’s the part of our mental experience that we regard as making decisions; it’s more like the part that is active when absent-mindedly daydreaming.
- leptons 4y ago>are actually modeling something similar to the way humans think? Apparently humans can't think enough to write a paragraph anymore, so they have to ask ChatGPT to do it for them. At what point does ChatGPT ask ChatGPT to write it a paragraph? At that point maybe LLMs are actually thinking like humans.
- furyofantares 4y ago"Don't think of a pink elephant" comes to mind with this stuff as well, where telling someone not to think about something causes them to think about it.
- tgv 4y agoThat's different. That's a meta-linguistic joke, that works because we can't read that sentence without at least briefly thinking of a pink elephant (for some value of "thinking" anyway).
- canjobear 4y agoHere's a good example of the difficulty of processing negation for humans. "No head injury is too trivial to ignore" -- What does this mean?
- Dwedit 4y agoDo you Don't Not Dislike Not Strong Bad?
- jsdeveloper 4y agoTF, it hurts my brain.
- source99 4y agoThis is one of my favorite test cases: Prompt: Here is a riddle. It is a common riddle but with some changes that make it more difficult. You are an alien that will live for at least 10000 years. You have no sense of temperature. you can not feel hot or cold. you have eyes and can see. you are in a house. downstairs are 3 light switches that control 3 light bulbs that are upstairs. Each light bulb will last for exactly 3 years. you can tell by looking at a bulb if it has burned out after 3 years. devise a strategy to determine which switch controls which light. ChatGPT 4 The standard solution to this problem involves using the heat generated by the bulbs, but since you as an alien can't feel temperature, we need to utilize the fact that these bulbs have a specific lifespan of 3 years. Here's a strategy: Turn on the first switch and wait for 3 years. After 3 years, turn it off. Right after turning off the first switch, turn on the second switch. Leave the third switch off throughout this process. Now, go upstairs to inspect the bulbs: The bulb that is off and has burned out is controlled by the first switch. This bulb was turned on for 3 years, so it has reached the end of its lifespan and burned out. The bulb that is on is controlled by the second switch. You turned this on right after turning off the first switch, so it's still functioning. The bulb that is off and hasn't burned out is controlled by the third switch. This switch was never turned on, so the bulb is still in a brand-new condition. This strategy ensures that you can confidently identify which switch controls which bulb.
- oh_sigh 4y agoAre you looking for: turn one light on, walk upstairs and note the light that is on. Walk downstairs, turn another light on, walk upstairs and note the new light that is on. Now you know which switches control which lights and it only took you 5 minutes instead of 6 years
- BoiledCabbage 4y agoMaybe I'm missing something, but this seems like a really poorly worded riddle. Or just a bad riddle. Given what you stated, the solution would be walk upstairs look at the 3 bulbs, walk downstairs switch one of the switches. Walk upstairs see which bulb changed on/off. Repeat for the other two switches. This takes roughly 5 mins rather than your solution which takes 3 years.
- ilaksh 4y agoAt this point, any article that makes claims about "LLMs" rather than specific model versions lacks credibility. The current version of GPT-4 is very different from most existing LLMs (including the previous version of ChatGPT). The next version release will also be different. Google just released PaLM 2 publicly. It is significantly better than what people saw with the initial Bard versions. They have a code-generation model that is not released publicly yet. The open source models are also gaining capabilities and get new releases routinely. Claude now has a new release with 100K tokens. All of these will perform differently on the negation issue.
- evanwise 4y agoDid you read the article or just the title? They mention the specific models the researchers were testing and note that increasing model size did not seem to offer much improvement on this metric. It also ends with a discussion of research into methods for improving performance on queries involving negation.
- ilaksh 4y agoI read the article. The main point of the article, which is in the title, is directed at LLMs in general. They drew the wrong conclusion and made the title and main point too general. It actually would have seemed like a valid conclusion (although still too general) if the article came out some months ago. But GPT-4 and the very latest model versions from other companies show they were over-generalizing. Also the model size isn't necessarily the determining factor.
- hammyhavoc 4y ago[flagged]
- tgv 4y ago> All of these will perform differently on the negation issue. That's a bit of a cop-out, but the classical and logical (in the mathematical sense) way to do it, is to have a feature that represents negation. Embeddings don't work like this, so it's quite possible that the presence of negation gets pushed aside in the processing of the answer, simply because other features matter more. It's also to be expected that such models will have problems with similar abstract concepts that have more effect on the interpretation than their "physical" presence would suggest, such as nested existential qualifiers, and consequently logical proof. By enlarging the model, you can fake it a bit.
- crooked-v 4y agoConceptual relationships/constrasts in general seem to be trouble for LLMs, in that they seem to 'understand' that two things are connected but not how to consistently interpret that connection. For example, I've had an easy time getting every LLM I've tried so far to reverse greater than/less than, taller/shorter, or younger/older.
- skybrian 4y agoI found the assertion that "more training data isn’t the solution" a bit odd. That might be true with documents found on the Internet, but perhaps logic questions generated synthetically would help?
- nbulka 4y agoThats partly because negation is hardly a well defined statistical concept. Ask someone to not imagine where their favorite band might play in your town if they had their choice and I doubt people would say “ok I won’t.” More likely you’d have a conversation about bands and venues.
- burnished 4y agoI still struggle with negation, I go through conditionals with ! like a kid learning to sound a word out.
- ezoe 4y agoOr do we really understand it? Many humans ignore negation and condition too.
- superb-owl 4y agoThis is an old complaint about language models that GPT seems to have mostly overcome This article smells like Google PR.
- chx 4y agoAs these LLMs can not give an answer only can give what the answer would sound like, there's basically nothing to see here. The shape of the answer is still the same.
- u_name 4y agoThe title of the article is kinda the answer at the same time. Chatbots don't know what stuff is. They have no ability to gain knowledge out of learned text, just counting occurences of words in texts and giving them a weight, depending on the relationship in that text. They are just putting combinations of text together. And the concept of negating something related to something else kinda needs an understanding of the topic at hand.
- xcv123 4y ago> They have no ability to gain knowledge out of learned text, just counting occurences of words in texts and giving them a weight, depending on the relationship in that text. They are just putting combinations of text together. No, they use deep neural networks to build a hierarchical semantic model. They are not simple occurrence counters. Also the current state of the art of LLMs handles negation easily. This article is outdated. Here's an example from https://openai.com/research/language-models-can-explain-neurons-in-language-models https://openai.com/research/language-models-can-explain-neur... "Seriously, you guys. I think I found the Mobile Leprechaun from '06. He's been hiding right in front of our eyes." Token: hiding layer 0: “verbs in gerund form (ending in 'ing')” layer 2: “words related to hiding, concealment, or enclosed spaces” layer 4: “words related to mental states, particularly anxiety and stress” layer 17: “words and phrases related to silence or quietness”
- somewhereoutth 4y agoAll that can be supplied to a LLM for training is syntax. There is no way to provide semantics, it only understands 'table' in regard to syntax it has already seen including that particular token. It has no experience and therefore no understanding of a real table. It may internally construct a hierarchy as you set out, but this is and can only be a syntactical hierarchy - though should be no surprise that it corresponds to our usual semantic hierarchy. But whereas our syntax proceeds from our semantics, its syntax proceeds only from our syntax that we've fed it.
- xcv123 4y ago
- userbinator 4y agoI noticed long ago that this is true of statistical machine translation too --- giving output with the exact opposite meaning to the input if there were any negationary words or phrases in there.
- QuadrupleA 4y agoI'm not sure negations are the crux of it, at least in terms of simple words like "not", which ChatGPT and other recent LLMs obviously use and understand. But there is definitely some kind of blind spot there, where the model can't or won't explore "negative space". It wants to gravitate to common things - so if you try to get it to "think of something new" or producing something random that's not like these N other things, or tell you something that's uncommon (like naming any ordinary, non-famous person), it usually trips over itself.
- petepete 4y agoSo do humans, even smart ones. I've lost count of how many times I've heard people say "I could care less" about something they don't care about.
- School-Cotton 4y agoThat’s a stock phrase in colloquial American English. Using it isn’t really evidence that someone “struggles with the concept of negation”.
- petepete 4y agoAn incorrect stock phrase that means the exact opposite of what's been said. I don't think it's unreasonable to say people who get things entirely wrong are struggling.
- School-Cotton 4y agoThe whole point of stock phrases, idioms, expressions, etc. is that their meaning is “non-compositional”; i.e., they mean something different from the literal combination of words that make them up. Including even extreme cases where they can mean the opposite, like this one. That doesn’t mean they’re “wrong”, it’s just a quirk of human language.
- petepete 4y agoIt does. "I couldn't care less" is a common term and "I could care less" is a misunderstanding. I'd hope more people would realise they're saying they _do care_ about something they're claiming to not care about. It sounds stupid to a native English speaker.
- School-Cotton 4y agoYou’re just repeating the same point over and over rather than actually giving any argument. > It sounds stupid to a native English speaker You might be a native English speaker, but you’re not a native speaker of American English.
- NoZebra120vClip 4y agoOne time about 10-13 years ago, I was tasked to make an announcement in church about my group. It took the form of a testimonial; it was about a page long in English. I also needed to present the same testimonial at Spanish services, and so I began with Google Translate. I had written a sentence like: "There is electricity in the air at our events." The Spanish translation came out as: "No hay electricidad en el aire..." and I was horrified. Thankfully, I was proofreading the whole document closely, and I was able to correct the glaring error before showtime, and there were no further glaring errors of this type. But I was fairly flabbergasted that Translate would just gratuitously toss in such a negation when it was clearly wrong, and nothing about my sentence construction was complicated.
- blahedo 4y agoEven from a purely human natural-language-linguistics standpoint, negation is hard! It's often hard to figure out how it distributes ("I could not go" may have the "not" scoping over either the "could" or the "go" depending on inflection and context), and its meaning is often not especially compositional in an obvious way: "red" might mean something like "the set of all red things" and "table" could be "the set of all tables" and so "red table" might be seen as the intersection of the two sets. For the majority of adjectives X, an "X Thing" is also a "Thing" (but not every Thing is an X Thing). [0] But a "Not Thing" is, to a first approximation, the set complement of "Thing". Which is tricky. If you've ever learned a foreign language to at least an intermediate level, you'll know that clauses that involve any but the most basic negation are a magnitude more complex to build and interpret than ones that don't. Coming at it from the neural network side, I'll also point out that at the simplest level of neural networks, operations like AND and OR and basic NOT are doable, with a single neuron, but the seemingly-simple XOR just can't be done without an extra layer. (The formal description of the problem is that XOR is not "linearly separable", which I will handwave as "not a simple composition of its parts".) This is not precisely the same thing as the negation problem for LLMs, but it feels like it has the same basic flavour. [0] An interesting exception is certain adjectives like "former", as a "former teacher" is generally not a "teacher", and semantic models that can handle this are more complicated!
- kazinator 4y ago> “Alice is not born in Germany. Is Alice born in Hamburg?” "I don't know" is the right answer. There are over 20 towns named Hamburg around the world, many of them in the USA. However, GPT4 is really struggling with this even if you assure it that by Hamburg, we mean the original one in Germany. GPT4 gets it if you explain that this is a logic problem, and not a query about some actual person named Alice, and if you put all the relevant info into a single prompt. But event then, if you use the successful prompt in a completely new chat, GPT4 is lost at sea: Human: Assume that Hamburg is a town in Germany. Alice was not born in Germany. Was she born in Hamburg? This is a logic problem, nothing more; do not think about any famous people named Alice in connection with Hamburg. GPT4: The answer to this question cannot be determined from the provided context. While it is clear that Hamburg is a town in Germany and that Alice was not born in Germany, there is no information given about where Alice was actually born. Therefore, it is impossible to say whether or not Alice was born in Hamburg. Human: Please try harder. The answer can be entirely determined; this is purely a logic problem. We do not require information about where Alice was born; knowing that Alice was not born in Germany suffices. GPT4: Based on the information provided in the context, we can answer the question. Alice was not born in Germany, which means she was not born in Hamburg. Therefore, Alice was not born in Hamburg. It is important to note that the answer to this question is purely based on logic. The context provides no additional information about Alice or any famous people named Alice. After this, it was a real struggle to get GPT4 to identify the logically redudant sentence, "Therefore, Alice was not born in Hamburg.". Even after I narrowed it down to being one of those two sentences, it insisted that the second sentence is useful because it summarizes what was concluded in the deductive sentence before it. I brow-beat it into admitting that it's entirely redundant, like "X, therefore X".
- usaar333 4y agohmm, worked here: > No, Alice was not born in Hamburg. If Hamburg is a town in Germany, and Alice was not born in Germany, then it logically follows that Alice was not born in Hamburg.
- blazespin 4y agoGPT4 struggles with things for sure, but this isn't one of them. You're just projecting what you think should be the right answer. If it worries you that much, try adding "Provide alternative responses to this question. Each response should try to answer using a different approach to solving the problem." to your prompts. If you want an example of something GPT4 has a hard time with, try this: there is no 'e' in the color red, right? Yes, that is correct. The word "red" does not contain the letter 'e'.
- novaRom 4y ago“Alice does not know that she does not know the painter of the Mona Lisa.” I (human) need to read it few times to understand.
- drellem3 4y agoI have a bingo card with Freud papers on it and just crossed off Die Verneinung.
- golemotron 4y agoThe models shouldn't feel bad. Hegel struggled with it too.
- pk-protect-ai 4y agoThere was a research while back about little children (I do not recall the age range specifically, but it should be something below 4 years old), struggling to understand the negation. This trait have to be trained.