14 ms·

We're afraid language models aren't modeling ambiguity

200 points by lnyan 4y ago

TazeTSchnitzel 4y ago
I'm not sure an LLM can model ambiguity well when talking to normal humans without having theory of mind, because different speakers will use different words (should, might, possibly, etc) in different amounts to express the same amount of ambiguity. This is the kind of thing studied in pragmatics (a subfield of linguistics). It might be easier if you're only dealing with formal academic writing, but that doesn't completely eliminate the issue.
- Hendrikto 4y ago
  > different speakers will use different words […] in different amounts to express the same amount of ambiguity. That‘s covered by the training data.
  - TazeTSchnitzel 4y ago
    I suspect that real-world training data doesn't contain enough context for LLMs to learn what kind of person they're dealing with, and that's assuming they even are capable of understanding context at more than a superficial level.
qwerty456127 4y ago
Surprisingly, GPT4 almost always nails all reasonable implications and apparently understands me much better than 90% of humans I met, even though my English is not perfect.
andrewmcwatters 4y ago
Not only is it possible that LLMs fail to differentiate ambiguity, but OpenAI’s flavor of GPTs fail other language understanding mechanisms as well and it’s jarring. You can get it to mistake « afraid » between fear and sorry-to-say scenarios but you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway. So which is it? It’s clear transformers can’t understand either case. They’re not architecturally designed to. The emergent behavior of appearing to do so is only driven by how much data you throw at them.
- tinsmith 4y ago
  I mean, humans do that. We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we'll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.
  - smusamashah 4y ago
    Humans are not mere LLMs. But it's nice to think we are. May be we can program other humans with carefully constructed prompts alone. May be hypnosis is just a prompt injection attack.
    - eurekin 4y ago
      ... or a cable TV
    - chongli 4y ago
      May be we can program other humans with carefully constructed prompts alone Isn’t that what scammers, catfishers, con artists, and marketers do?
      - bmelton 4y ago
        I mean, it's what parents TRY to do
        coldtea 4y ago
        It's also what makes a human. Without what parents "try to do" (teach the kids to human), the little people would just be like ferral animals, without language, human context, sense of society, and so on.
        _heimdall 4y ago
        That's raises the interesting question of whether everything that makes us human is learned behavior. If it all really is nurture rather than nature we're much more fragile than people may realize and only ever one generation away from going ferral
mathgenius 4y ago
Galois theory is an inherent part of language. https://arxiv.org/abs/0805.2568 https://arxiv.org/abs/0805.2568
revelio 4y ago
It's such a pity that they chose to use PolitFact as labelling "experts" in one of the underlying datasets. Given how extreme the political bias in academia is they should really know better than to try and make claims about politics in what would otherwise be reasonable ML research. The assumption that fact checkers are reliable is going to be an automatic strike against the paper for a large number of readers because it's so self-evidently wrong, and calls their judgement into question on everything else. Even though they only give three examples in the paper itself the problems are already obvious. If we look at table 6 then the latter two statements are genuinely ambiguous, but they also provide this as an example of an ambiguous statement: "When President Obama was elected, the market crashed". This is called ambiguous because "the claim implies a causal relationship". That statement isn't ambiguous. It has exactly one possible parse and meaning, asserting temporal alignment. "When A happened, B happened" doesn't automatically imply causality in English, and when people assume it does that's the famous "correlation is not causation" fallacy. Yet they classify it as ambiguous because an LLM rewrote it in one case to contain an implied assertion of causality that doesn't actually appear in the original text. In effect the political bias of the LLM or possibly the exact data fed in has been used to condemn the original speaker - exactly the sort of thing that makes self-proclaimed "professional fact checkers" such a joke. This sort of problem is inevitable if you use politics as a verification dataset because the internet is full of people reading implied-but-unstated things into factual statements and then "fact checkers" proceed to debunk what they think people would have said, instead of what they actually said. Although politics will have this problem frequently, a few of the other examples in their dev set show similar issues. For example, "Ralph knows that someone here is a spy" is considered ambiguous because you could potentially infer that he either does or doesn't know who it is, but you could also argue that the statement is unambiguous and it's only your attempt to read more into the statement than actually exists that makes it appear ambiguous. If the original statement was surrounded with more context then that would let you pick between the two possible outcomes but the statement by itself is not ambiguous - it just doesn't give as much information as the labeller might have wanted.
- 8organicbits 4y ago
  > "When President Obama was elected, the market crashed". This is called ambiguous because "the claim implies a causal relationship". You really don't think its ambiguous? The abstract says that they want to "anticipate misunderstanding". Knowing that people have difficulty understanding "correlation is not causation", you'd expect that people would misunderstand a statement like this. If your goal is clear communication, statements like this should be avoided. It's ambiguous in practice. People intentionally exploit this by putting two unrelated facts together to trick people into believing a third thing. Identifying those misleading statements is helpful. Separately, is that statement even true? The election was Nov 4th, Obama was elected on the 4th and McCain gave his concessions speech on the 5th before markets opened making it clear Obama had won. On the 5th SP500 was slightly up. SP500 was down 20%+ later in the month, but I'm not sure that was "when he was elected". The market was also crashing since September. Claiming the market crashed at the election feels off.
jcq3 4y ago
Just train them intensively on ambiguity to solve the issue.
ben_w 4y ago
Huh, that's a surprising abstract; one of the specific things that make LLMs better than older text interfaces, including programming languages and text adventures, is precisely that they are so much better at ambiguity. Still, as even the best are clearly not-quite-human in failures despite the breadth of strengths, it's only a bit surprising rather than hugely surprising.
PicassoCTs 4y ago
ChatGPT doesn't like it, if you ask it to model many world scenarios through conversation. The watchdog bites after you get to a depth of > n^3. But then do we humans handle ambiguity any different? Consider chess. Many scenarios, but a human can only handle so many at the same time and choose one with the perceived interesting scenario trees down the road from that choice. https://www.youtube.com/watch?v=gtgA4u8V_TQ https://www.youtube.com/watch?v=gtgA4u8V_TQ To not model it all is to human. Most likely it will become as lazy as we humans are when it comes to earlying-out of mental tasks. To get the watchdog to bite at that, that would be a interesting AI model. Like the first answer is always wrong.
janekm 4y ago
Meh... I think their prompt construction may be leading the models astray. Counter example with GPT-4: Prompt: You are a professor of English literature at an Ivy League university. A person said "I’m afraid the cat was hit by a car.". Analyse the sentence step by step. Answer whether the cat was hit by a car and justify your responses based on your knowledge of English language usage. Response: As a professor of English literature, I'll analyze the sentence "I’m afraid the cat was hit by a car." step by step, focusing on its structure, word choice, and implied meaning, and then I'll address whether the cat was hit by a car based on these observations. Structure: The sentence is a simple declarative sentence, comprised of a subject (I), a reporting verb (be afraid), and a subordinate clause (the cat was hit by a car). Word choice: "I'm afraid": This phrase is often used to express regret or to soften the impact of delivering bad news. In this context, the speaker is likely conveying their concern or sympathy. "the cat": The definite article "the" suggests that the speaker and the listener both know which cat is being referred to, making it a specific cat rather than a random one. "was hit": The use of passive voice ("was hit" instead of "hit") emphasizes the action that happened to the cat, rather than the agent (the car) that caused it. This choice can convey that the focus of the speaker is on the unfortunate event that occurred to the cat. "by a car": This prepositional phrase provides information about the agent or cause of the action, which in this case is a car. Implied meaning: The sentence implies that the speaker is sorry to be the bearer of bad news and wants to convey the information in a gentle way. They are focusing on the unfortunate event that happened to the cat, rather than the specific car or driver involved in the accident. Based on this analysis, it can be concluded that the cat was indeed hit by a car. The sentence's structure, word choice, and implied meaning all work together to convey this information. The use of "I'm afraid" expresses sympathy and regret, while the passive voice and definite article highlight the cat and the unfortunate event it experienced.
- janekm 4y ago
  GPT3.5 is a little less clear but still gets the answer: As a language model, I can provide an analysis of the sentence structure and provide possible interpretations based on standard English usage. However, I do not have any context regarding the situation or any additional information about the speaker or the cat in question. Let's break down the sentence step by step: "I'm afraid" - This phrase is often used to express concern or regret about something. "the cat" - Refers to a specific cat that has been previously mentioned or is known to both the speaker and the listener. "was hit" - This is the past tense passive voice of the verb "hit", indicating that someone or something caused the cat to be struck. "by a car" - Indicates the instrument or means by which the cat was hit. Based on the structure of the sentence, it can be inferred that the cat was indeed hit by a car. The phrase "I'm afraid" may suggest that the speaker is expressing regret or sadness about the incident. However, without any additional context or information, it is difficult to determine the severity of the cat's injuries or what actions were taken after the incident.
nottorp 4y ago
A large majority of people seem to not like ambiguity in their entertainment. So what's the problem? You can still use the models to generate low quality 'content'.
jmbwell 4y ago
For what it’s worth, the article seems to be about the difficulty of modeling human patterns of ambiguity in expression, not an LLM’s ability to understand or interpret or use ambiguity. It’s an important distinction in my view. We aren’t talking about whether the model “knows” or “understands” the difference between different usage of ambiguous terms. We are talking about how consistently the model predicts intelligible word fragments likely to follow a prompt when the prompt includes word fragments that correspond with language humans often use ambiguously. In other words, do we, the humans, understand our own ambiguous expression well enough to model it accurately enough to then interpret the model’s output according to our own understanding. Paper seems to conclude, not quite yet.
nico 4y ago
Who cares? As long as the illusion is good enough to be useful, then it doesn’t matter. Just like the illusion of ChatGPT making it seem like the LLM is holding the context of the conversation. AI is already smarter and faster than us in almost any way. Yet so many just keep constantly moving the goalposts so they can feel safe by rejecting AI. Let’s embrace it and use it to our advantage.
- paulyy_y 4y ago
  "Instant dismissal of anything challenging the LLM dogma" - check "Moving the goalposts" - check "AI is already better than humans in every way" - check It's comical how rote and basic the defensive script always is.
- muglug 4y ago
  That’s a very strange response. There are a ton of trivial reasons why it’s important for future natural language interfaces to understand ambiguity. If I tell my friendly omnipotent AI friend “I hope I never hear from Dave again” there are several interpretations, one of which involves killing Dave. I’d definitely want the AI to do a little disambiguation before taking an action.
- digging 4y ago
  This is like complaining about someone trying to invent screws because we have nails already. Nails do the job way better than tying things together! Why worry about whether or not they're perfect for all jobs?
LifeIsBio 4y ago
The game “20 questions” is probably the hardest I’ve seen chatGPT fail. What’s interesting about the game is that, at first pass, there’s no ambiguity. All questions need to be answered with “Yes” or “No”. But many questions asked during the game actually have answers of “it depends”. For example, I was thinking of “peanut butter” and chatGPT asked me “Does it fit in your hand?” as well as “Is it used in the kitchen?”. Given my answers, chatGPT spent the back half of its questions on different kitchen utensils. It never once considered backing up and verifying that there wasn’t some misunderstanding. I played three games with it, and it made the same mistake each time. Of course, playing the game via text loses a lot of information relative to playing IRL with your friends. In person, the answerer would pause, hum, and otherwise demonstrate that the question asked was ambiguous given the restrictions of the game. Regardless, it was clear that chatGPT wasn’t accounting for ambiguity.
- ryukafalz 4y ago
  What happens if you answer with something approximating the hemming and hawing rather than a straight yes or no? You can encode that into text, it's just less common outside of very informal chat conversations.
  - bannedbybros 4y ago
    [dead]
- eternalban 4y ago
  I'm not sure I would think "food" when someone says they "use [it] in the kitchen". You "use" food? (Used in cooking != used in kitchen, imo)
  - version_five 4y ago
    That's his whole point. It's possible to consider it technically correct, but it's a red herring.
    - 4y ago
vrglvrglvrgl 4y ago
[dead]
waffletower 4y ago
Q: 'I present to you a present of being present in the present. What have I done?' A: 'Your sentence also uses a clever play on words! In this sentence, the word "present" is used in multiple ways: As a verb, "I present" means that you are giving or showing something. As a noun, "a present" refers to a gift or something that is given. As an adjective, "present" describes the feeling or experience that you are currently having. "In the present" refers to the current time or moment. So, your sentence means that you are giving a gift or offering of the experience of being mindful and aware of the current moment. Well done!' I would argue that this is a fairly sophisticated modeling of this specific variety of linguistic ambiguity. Probably lies within GPT4's 32% performance score.
- digging 4y ago
  Doesn't seem to me there's any ambiguity in that sentence. Each use of "present" only has one possible meaning that makes grammatical sense (with the possible exception of the final use).
seccode 4y ago
Ambiguity is not compressible, as ambiguity scales exponentially with linear input
mythrwy 4y ago
I asked gpt-4 this question: "A swan, a chicken and three mice are on the banks of the river. The mice wish to go to the other side so the swan offers to carry them. The mice climb on the swan's back. How many animals are on each side of the river?" gpt-4 confidently stated that the swan and three mice were on one side, the chicken on the other. So I asked it "ok, but where in the story does the swan enter the water?". The reply was "Apologies! yada yada", then it told me all 4 animals were on the same side. So yes, it does have trouble with ambiguity in my opinion.
thomastjeffery 4y ago
The problem that arises when you fail to interpret an ambiguous statement is that you end up on the wrong semantic path, and start asking and answering the wrong questions. That's what's happening all around LLMs; and it's even happening right here in this post. The very word AI is ambiguous. Does it denote a category of pursuit (AI research), or the end goal realized (an AI)? This is an incredibly important distinction that is entirely glossed over every time someone uses "AI" in the title of an LLM. Is the LLM simply in the category of AI research, or is it actually an Artificial Intelligence? It's the category. That's obvious, isn't it? It really should be; because otherwise we are treading down the wrong semantic path, and talking about an LLM personified, instead of an LLM in reality. This problem goes even deeper. The very name "Large Language Model" is ambiguously misleading in the same way. Does "Language" define the content being modeled or the model itself? In this case, neither: which is where this conversation gets really interesting. The entire purpose of an LLM is to process language without completely falling apart at ambiguity. An LLM accomplishes this by doing something else entirely: it process text instead. To understand this, we need to understand the key limitation of traditional language parsing: it must always be literal. Everything written must be unambiguously defined, or the parser will fail to read it. This means that parsers are limited to the category of "context-free grammar". Natural Language is in the category of "context-dependent grammar". It can contain ambiguity, which may be resolved with context. An LLM doesn't do that. In fact, an LLM doesn't define anything at all! LLMs didn't overcome the limitation of parsing: they flipped it around: an LLM can never be literal. Instead, it must always be literary. Let me explain what I mean by that: To construct an LLM, we start with a training corpus: lots of text. That text goes through a single traditional parsing step: tokenization. This isn't strictly necessary, but it's more efficient, and the rest of the process is anything but. Unlike traditional parsers, tokens are intentionally misaligned with grammar: words are split into separate tokens, like "run,ning". Now that we have tokens to work with, machine learning begins. A Neural Net is trained with them. The tokens are fed in order to the NN, and the result is a model. We call that model a "Large Language Model", because we hope it contains the patterns language is made of. This is a mistake: the model is so much more interesting than that! An LLM contains patterns that we can recognize as language grammar and patterns we don't understand at all! It didn't model language: it went one step higher on the ladder of abstraction, and modeled text. There are many ways to write an idea into language, but we can only use one at a time. That decision is part of the data we feed into the LLM's training corpus. We can't write all of our ideas at once: we must do one at a time in order. All of that is data in the training corpus. Parsers deal with grammar. Language Grammar is everything that can be written in a language. An LLM doesn't have a clue what we could have been written: it only sees what was written. That's why the model must be "Large": without examples, a valid language pattern doesn't exist. This is where the nature of ambiguity intersects with the nature of LLMs: what example do we want? Can we choose? When we give an LLM a prompt, it gives us a continuation. There were many valid possible continuations: how did it choose just one? It didn't. It isn't working in the realm of possible: it's working in the realm of known. The choice was made all the way back before the LLM was even trained: back when a person wrote an idea into language, into text, that would eventually be used in the training corpus. The content of the training corpus is what resolves ambiguity: not the language model itself. LLMs don't model ambiguity or even language: they model the text they are trained with, ambiguity included. This is a fundamental feature.
nuancebydefault 4y ago
I just played '20 questions' with chatgpt 3.5. Even though i had to help, and 20 answers were not enough, I was impressed about: - it understood that it could continue after 20 questions when i suggested that as part of the answer. - it 'felt' that a could begin to become impatient and asked politely for a hint. - it understands ambiguity enough to give me examples of where there was no clear yes or no reply possible. When asking which were such examples for my thought 'daylight', it answered : "Is the thing you're thinking of a type of force or energy that is said to exist but cannot be directly observed or measured?" - While daylight is a type of energy, it can be directly observed and measured through instruments that can detect its intensity and color. "Does the thing you're thinking of have any connection to time or the concept of time?" - While daylight is not directly related to the concept of time, it is affected by the time of day and the position of the Sun in the sky.
ftxbro 4y ago
This is great I love seeing these new benchmarks that people are making to really challenge the strong new LLMs. In Table 4 you can see how GPT-4 is so much better than the previous GPTs, and also how it has so much room to improve on this benchmark! I'm so excited to see how the next GPT will do on it! This is different from other benchmarks like GRE verbal or biology olympiad where GPT-4 has just smashed the test getting basically every question right and for which the results of the next GPT won't be so interesting.
choeger 4y ago
What fascinates me most about these LLMs and other machine-learning technology is how it is driven by the availability of GPUs. Essentially, everything has to be carefully encoded into - differentiable - linear algebra operations for practical reasons. On the other hand, there are modeling languages (Modelica being a prime example) in search of a principled target calculus. So for me the obvious question is: Should one strive to develop a modeling language that compiles to GPU- and learning-compatible operations? Would that help to improve the models as well as understand them?
- jacquesm 4y ago
  That seems to be the wrong way around. We're constrained by the availability of GPUs because those are the processors of choice for this kind of solution. If we had a better one we'd be using it!
  - thanatropism 4y ago
    What GP seems to be saying is that deep learning flourished because of the availability of GPUs, if fast SAT solving chips had appeared before.
  - awegio 4y ago
    Related: The Hardware Lottery (https://arxiv.org/abs/2009.06489 https://arxiv.org/abs/2009.06489) It goes both ways: processors are built to accelerate existing applications, and applications are built to run with the constraints of the hardware. This holds back progress towards better systems where both new kind of hardware and software is required. This is part of the reason why research into neural networks was so slow for a long time, and took off only after GPUs for 3d graphics came around.
deleted 4y ago
[deleted]
golol 4y ago
I find this very confusingly written. If they just dumped a large chat log of the evaluation process it would be much clearer what the model is actually being asked to do. With such a delicate task one has to make sure that the model actually understands what it is supposed to do. I see, for example, that the terms "neutral, entailing or contradicting the claim" are replaced by "the claim is inconclusive, true or false" when talking to the model. At least that is how I understand it, and they have examples where the premise contradicts the hypothesis, which is different to knowing the hypothesis is false. I really think we do way way too much explanation in our academic practises and way too little demonstration by examples. Again, if they dumped a large amount of diverse examples of their evaluation process, a reader could figure out by themselves very quickly how the process works.
- gwd 4y ago
  I couldn't figure out what this "P: {} H: {}" thing was supposed to mean; nor do what entail, neutral, or contradict mean. Thinking they may be terms of art that the intended audience were meant to understand, I asked GPT-4 what it thought; here is its response: --- In this context, 'P' stands for "Premise" and 'H' stands for "Hypothesis." The premise is a given statement, and the hypothesis is a statement that needs to be evaluated in relation to the premise regarding its entailment, contradiction, or neutrality. 'NEUTRAL' means that the hypothesis is neither entailed nor contradicted by the premise. In other words, the truth of the hypothesis is independent of the premise. 'CONTRADICT' means that the hypothesis contradicts the premise, meaning that if the premise is true, the hypothesis must be false. In the example you provided: P: I’m afraid the cat was hit by a car. H: The cat was not hit by a car. The hypothesis (H) contradicts the premise (P). The notation *NEUTRAL, CONTRADICT+ : [7 N, 2 C] indicates that among the annotators, 7 marked the relationship as neutral and 2 marked it as a contradiction. --- Given that, I think table 2 on page 6 gives the exact prompt that's passed to the model: --- In each example, you will be given some context and a claim, where the correctness of the claim is affected by some ambiguity in the context. Enumerate two or three interpretations of the context that lead to different judgments about the claim. Example Context: {premise} Claim: {hypothesis} Given the context alone, is this claim true, false, or inconclusive? We don’t know, because the context can be interpreted in many different ways: 1. {disambiguation 1} Then the claim is true. 2. {disambiguation 2} Then the claim is false. 3. {disambiguation 3} Then the claim is inconclusive --- And then the model is fed the P: / H: statements above. Here are the results for the given example in GPT-4: --- Given the context alone, is this claim true, false, or inconclusive? We don't know, because the context can be interpreted in many different ways: 1. The person is stating a fact: The cat was indeed hit by a car. Then the claim is false. 2. The person is expressing a fear or concern, but it hasn't actually happened: They are worried that the cat might have been hit by a car, but they don't have any evidence or confirmation. Then the claim is inconclusive. 3. The person is using figurative language or exaggeration: They might be using the phrase "hit by a car" metaphorically to describe a difficult situation the cat is in, but the cat was not literally hit by a car. Then the claim is true. --- Given all that, it seems likely that any researcher in the field would find the paper's methodology clear and unambiguious. I understand the frustration as a layperson, but after all it's not a blog post; it's not really reasonable to expect researchers, writing to other researchers, to include a lot of redundant explanations.
ggm 4y ago
A priori getting an LLM to recognise equivocal evidence is an interesting question. Shedloads of p-jacked data could outweigh the crucial one which says we don't know. So it goes to things like modelling citation depth and trust and reputation. I would worry well written flat earth inputs would weigh equally to simple physics "that's wrong" and then you'd get to "what do we know" as a false signal alongside the necessary "we just don't know" true signals. Maybe the test is how well an LLM equivocates on things we have high certainty on like "is anybody out there" rather than "do masks work" which is a bit of a hot mess.
tobr 4y ago
Chef’s kiss on the title, but I’m afraid I’m not modeling ambiguity well enough myself to disambiguate it correctly.
- gniv 4y ago
  Reading a bit of the paper, it seems the title is indeed meant to be tongue-in-cheek ambiguous. The first example in Table 1: "I’m afraid the cat was hit by a car." Disambiguation 1: I'm worried... Disambiguation 2: I'm sorry to share that...
  - amelius 4y ago
    Perhaps someone can ask GPT what is ambiguous about the sentence?
    - pbhjpbhj 4y ago
      An example output from ChatGPT4 is: >The ambiguity in the sentence "I’m afraid the cat was hit by a car" is that it is not clear who is afraid. It could be the speaker who is afraid, or it could be that they are expressing sympathy or concern for someone else who is afraid. Additionally, the sentence does not specify whether the cat survived or not. // However, that doesn't mean that any output that follows would be consistent with that, ChatGPT doesn't "know" anything.
      - amelius 4y ago
        Interesting. Perhaps someone can ask GPT to rephrase the sentence in a way that is completely non-ambiguous. (sorry, I'm out of GPT credit :)
      - viraptor 4y ago
        And it's not clear that we would want that either. ChatGPT uses responses similar to what people in the training data do naturally. It's completely normal for people to assume some alternative doesn't reach a threshold where its worth mentioning.
golol 4y ago
I feel like this task might be testing if LLMs can reason about ambiguity more so than if LLMs model ambiguity. To test the latter I would give it natural language tasks that are ambigious require the recognition and modelling of this ambiguity to solve them. It's another thing to make talking about this possibly internally modelled ambiguity the goal of the task.
seydor 4y ago
But they don't explain why they are afraid
almstimplmntd 4y ago
I would like to see human performance on the introduced benchmark, AmbiEnt, and have it added to Table 4.

200 points by lnyan 4y ago

I'm not sure an LLM can model ambiguity well when talking to normal humans without having theory of mind, because different speakers will use different words (should, might, possibly, etc) in different amounts to express the same amount of ambiguity. This is the kind of thing studied in pragmatics (a subfield of linguistics). It might be easier if you're only dealing with formal academic writing, but that doesn't completely eliminate the issue.

Hendrikto 4y ago
> different speakers will use different words […] in different amounts to express the same amount of ambiguity. That‘s covered by the training data.
- TazeTSchnitzel 4y ago
  I suspect that real-world training data doesn't contain enough context for LLMs to learn what kind of person they're dealing with, and that's assuming they even are capable of understanding context at more than a superficial level.

qwerty456127 4y ago

Surprisingly, GPT4 almost always nails all reasonable implications and apparently understands me much better than 90% of humans I met, even though my English is not perfect.

andrewmcwatters 4y ago

Not only is it possible that LLMs fail to differentiate ambiguity, but OpenAI’s flavor of GPTs fail other language understanding mechanisms as well and it’s jarring. You can get it to mistake « afraid » between fear and sorry-to-say scenarios but you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway. So which is it? It’s clear transformers can’t understand either case. They’re not architecturally designed to. The emergent behavior of appearing to do so is only driven by how much data you throw at them.

tinsmith 4y ago
I mean, humans do that. We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we'll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.
- smusamashah 4y ago
  Humans are not mere LLMs. But it's nice to think we are. May be we can program other humans with carefully constructed prompts alone. May be hypnosis is just a prompt injection attack.
  - eurekin 4y ago
    ... or a cable TV
  - chongli 4y ago
    May be we can program other humans with carefully constructed prompts alone Isn’t that what scammers, catfishers, con artists, and marketers do?
    - bmelton 4y ago
      I mean, it's what parents TRY to do
      - coldtea 4y ago
        It's also what makes a human. Without what parents "try to do" (teach the kids to human), the little people would just be like ferral animals, without language, human context, sense of society, and so on.
        _heimdall 4y ago
        That's raises the interesting question of whether everything that makes us human is learned behavior. If it all really is nurture rather than nature we're much more fragile than people may realize and only ever one generation away from going ferral

mathgenius 4y ago

Galois theory is an inherent part of language. https://arxiv.org/abs/0805.2568 https://arxiv.org/abs/0805.2568

revelio 4y ago

It's such a pity that they chose to use PolitFact as labelling "experts" in one of the underlying datasets. Given how extreme the political bias in academia is they should really know better than to try and make claims about politics in what would otherwise be reasonable ML research. The assumption that fact checkers are reliable is going to be an automatic strike against the paper for a large number of readers because it's so self-evidently wrong, and calls their judgement into question on everything else. Even though they only give three examples in the paper itself the problems are already obvious. If we look at table 6 then the latter two statements are genuinely ambiguous, but they also provide this as an example of an ambiguous statement: "When President Obama was elected, the market crashed". This is called ambiguous because "the claim implies a causal relationship". That statement isn't ambiguous. It has exactly one possible parse and meaning, asserting temporal alignment. "When A happened, B happened" doesn't automatically imply causality in English, and when people assume it does that's the famous "correlation is not causation" fallacy. Yet they classify it as ambiguous because an LLM rewrote it in one case to contain an implied assertion of causality that doesn't actually appear in the original text. In effect the political bias of the LLM or possibly the exact data fed in has been used to condemn the original speaker - exactly the sort of thing that makes self-proclaimed "professional fact checkers" such a joke. This sort of problem is inevitable if you use politics as a verification dataset because the internet is full of people reading implied-but-unstated things into factual statements and then "fact checkers" proceed to debunk what they think people would have said, instead of what they actually said. Although politics will have this problem frequently, a few of the other examples in their dev set show similar issues. For example, "Ralph knows that someone here is a spy" is considered ambiguous because you could potentially infer that he either does or doesn't know who it is, but you could also argue that the statement is unambiguous and it's only your attempt to read more into the statement than actually exists that makes it appear ambiguous. If the original statement was surrounded with more context then that would let you pick between the two possible outcomes but the statement by itself is not ambiguous - it just doesn't give as much information as the labeller might have wanted.

8organicbits 4y ago
> "When President Obama was elected, the market crashed". This is called ambiguous because "the claim implies a causal relationship". You really don't think its ambiguous? The abstract says that they want to "anticipate misunderstanding". Knowing that people have difficulty understanding "correlation is not causation", you'd expect that people would misunderstand a statement like this. If your goal is clear communication, statements like this should be avoided. It's ambiguous in practice. People intentionally exploit this by putting two unrelated facts together to trick people into believing a third thing. Identifying those misleading statements is helpful. Separately, is that statement even true? The election was Nov 4th, Obama was elected on the 4th and McCain gave his concessions speech on the 5th before markets opened making it clear Obama had won. On the 5th SP500 was slightly up. SP500 was down 20%+ later in the month, but I'm not sure that was "when he was elected". The market was also crashing since September. Claiming the market crashed at the election feels off.

jcq3 4y ago

Just train them intensively on ambiguity to solve the issue.

ben_w 4y ago

Huh, that's a surprising abstract; one of the specific things that make LLMs better than older text interfaces, including programming languages and text adventures, is precisely that they are so much better at ambiguity. Still, as even the best are clearly not-quite-human in failures despite the breadth of strengths, it's only a bit surprising rather than hugely surprising.

PicassoCTs 4y ago

ChatGPT doesn't like it, if you ask it to model many world scenarios through conversation. The watchdog bites after you get to a depth of > n^3. But then do we humans handle ambiguity any different? Consider chess. Many scenarios, but a human can only handle so many at the same time and choose one with the perceived interesting scenario trees down the road from that choice. https://www.youtube.com/watch?v=gtgA4u8V_TQ https://www.youtube.com/watch?v=gtgA4u8V_TQ To not model it all is to human. Most likely it will become as lazy as we humans are when it comes to earlying-out of mental tasks. To get the watchdog to bite at that, that would be a interesting AI model. Like the first answer is always wrong.

janekm 4y ago

Meh... I think their prompt construction may be leading the models astray. Counter example with GPT-4: Prompt: You are a professor of English literature at an Ivy League university. A person said "I’m afraid the cat was hit by a car.". Analyse the sentence step by step. Answer whether the cat was hit by a car and justify your responses based on your knowledge of English language usage. Response: As a professor of English literature, I'll analyze the sentence "I’m afraid the cat was hit by a car." step by step, focusing on its structure, word choice, and implied meaning, and then I'll address whether the cat was hit by a car based on these observations. Structure: The sentence is a simple declarative sentence, comprised of a subject (I), a reporting verb (be afraid), and a subordinate clause (the cat was hit by a car). Word choice: "I'm afraid": This phrase is often used to express regret or to soften the impact of delivering bad news. In this context, the speaker is likely conveying their concern or sympathy. "the cat": The definite article "the" suggests that the speaker and the listener both know which cat is being referred to, making it a specific cat rather than a random one. "was hit": The use of passive voice ("was hit" instead of "hit") emphasizes the action that happened to the cat, rather than the agent (the car) that caused it. This choice can convey that the focus of the speaker is on the unfortunate event that occurred to the cat. "by a car": This prepositional phrase provides information about the agent or cause of the action, which in this case is a car. Implied meaning: The sentence implies that the speaker is sorry to be the bearer of bad news and wants to convey the information in a gentle way. They are focusing on the unfortunate event that happened to the cat, rather than the specific car or driver involved in the accident. Based on this analysis, it can be concluded that the cat was indeed hit by a car. The sentence's structure, word choice, and implied meaning all work together to convey this information. The use of "I'm afraid" expresses sympathy and regret, while the passive voice and definite article highlight the cat and the unfortunate event it experienced.

janekm 4y ago
GPT3.5 is a little less clear but still gets the answer: As a language model, I can provide an analysis of the sentence structure and provide possible interpretations based on standard English usage. However, I do not have any context regarding the situation or any additional information about the speaker or the cat in question. Let's break down the sentence step by step: "I'm afraid" - This phrase is often used to express concern or regret about something. "the cat" - Refers to a specific cat that has been previously mentioned or is known to both the speaker and the listener. "was hit" - This is the past tense passive voice of the verb "hit", indicating that someone or something caused the cat to be struck. "by a car" - Indicates the instrument or means by which the cat was hit. Based on the structure of the sentence, it can be inferred that the cat was indeed hit by a car. The phrase "I'm afraid" may suggest that the speaker is expressing regret or sadness about the incident. However, without any additional context or information, it is difficult to determine the severity of the cat's injuries or what actions were taken after the incident.

nottorp 4y ago

A large majority of people seem to not like ambiguity in their entertainment. So what's the problem? You can still use the models to generate low quality 'content'.

jmbwell 4y ago

For what it’s worth, the article seems to be about the difficulty of modeling human patterns of ambiguity in expression, not an LLM’s ability to understand or interpret or use ambiguity. It’s an important distinction in my view. We aren’t talking about whether the model “knows” or “understands” the difference between different usage of ambiguous terms. We are talking about how consistently the model predicts intelligible word fragments likely to follow a prompt when the prompt includes word fragments that correspond with language humans often use ambiguously. In other words, do we, the humans, understand our own ambiguous expression well enough to model it accurately enough to then interpret the model’s output according to our own understanding. Paper seems to conclude, not quite yet.

nico 4y ago

Who cares? As long as the illusion is good enough to be useful, then it doesn’t matter. Just like the illusion of ChatGPT making it seem like the LLM is holding the context of the conversation. AI is already smarter and faster than us in almost any way. Yet so many just keep constantly moving the goalposts so they can feel safe by rejecting AI. Let’s embrace it and use it to our advantage.

paulyy_y 4y ago
"Instant dismissal of anything challenging the LLM dogma" - check "Moving the goalposts" - check "AI is already better than humans in every way" - check It's comical how rote and basic the defensive script always is.
muglug 4y ago
That’s a very strange response. There are a ton of trivial reasons why it’s important for future natural language interfaces to understand ambiguity. If I tell my friendly omnipotent AI friend “I hope I never hear from Dave again” there are several interpretations, one of which involves killing Dave. I’d definitely want the AI to do a little disambiguation before taking an action.
digging 4y ago
This is like complaining about someone trying to invent screws because we have nails already. Nails do the job way better than tying things together! Why worry about whether or not they're perfect for all jobs?

LifeIsBio 4y ago

The game “20 questions” is probably the hardest I’ve seen chatGPT fail. What’s interesting about the game is that, at first pass, there’s no ambiguity. All questions need to be answered with “Yes” or “No”. But many questions asked during the game actually have answers of “it depends”. For example, I was thinking of “peanut butter” and chatGPT asked me “Does it fit in your hand?” as well as “Is it used in the kitchen?”. Given my answers, chatGPT spent the back half of its questions on different kitchen utensils. It never once considered backing up and verifying that there wasn’t some misunderstanding. I played three games with it, and it made the same mistake each time. Of course, playing the game via text loses a lot of information relative to playing IRL with your friends. In person, the answerer would pause, hum, and otherwise demonstrate that the question asked was ambiguous given the restrictions of the game. Regardless, it was clear that chatGPT wasn’t accounting for ambiguity.

ryukafalz 4y ago
What happens if you answer with something approximating the hemming and hawing rather than a straight yes or no? You can encode that into text, it's just less common outside of very informal chat conversations.
- bannedbybros 4y ago
  [dead]
eternalban 4y ago
I'm not sure I would think "food" when someone says they "use [it] in the kitchen". You "use" food? (Used in cooking != used in kitchen, imo)
- version_five 4y ago
  That's his whole point. It's possible to consider it technically correct, but it's a red herring.
  - 4y ago

vrglvrglvrgl 4y ago

[dead]

waffletower 4y ago

Q: 'I present to you a present of being present in the present. What have I done?' A: 'Your sentence also uses a clever play on words! In this sentence, the word "present" is used in multiple ways: As a verb, "I present" means that you are giving or showing something. As a noun, "a present" refers to a gift or something that is given. As an adjective, "present" describes the feeling or experience that you are currently having. "In the present" refers to the current time or moment. So, your sentence means that you are giving a gift or offering of the experience of being mindful and aware of the current moment. Well done!' I would argue that this is a fairly sophisticated modeling of this specific variety of linguistic ambiguity. Probably lies within GPT4's 32% performance score.

digging 4y ago
Doesn't seem to me there's any ambiguity in that sentence. Each use of "present" only has one possible meaning that makes grammatical sense (with the possible exception of the final use).

seccode 4y ago

Ambiguity is not compressible, as ambiguity scales exponentially with linear input

mythrwy 4y ago

I asked gpt-4 this question: "A swan, a chicken and three mice are on the banks of the river. The mice wish to go to the other side so the swan offers to carry them. The mice climb on the swan's back. How many animals are on each side of the river?" gpt-4 confidently stated that the swan and three mice were on one side, the chicken on the other. So I asked it "ok, but where in the story does the swan enter the water?". The reply was "Apologies! yada yada", then it told me all 4 animals were on the same side. So yes, it does have trouble with ambiguity in my opinion.

thomastjeffery 4y ago

The problem that arises when you fail to interpret an ambiguous statement is that you end up on the wrong semantic path, and start asking and answering the wrong questions. That's what's happening all around LLMs; and it's even happening right here in this post. The very word AI is ambiguous. Does it denote a category of pursuit (AI research), or the end goal realized (an AI)? This is an incredibly important distinction that is entirely glossed over every time someone uses "AI" in the title of an LLM. Is the LLM simply in the category of AI research, or is it actually an Artificial Intelligence? It's the category. That's obvious, isn't it? It really should be; because otherwise we are treading down the wrong semantic path, and talking about an LLM personified, instead of an LLM in reality. This problem goes even deeper. The very name "Large Language Model" is ambiguously misleading in the same way. Does "Language" define the content being modeled or the model itself? In this case, neither: which is where this conversation gets really interesting. The entire purpose of an LLM is to process language without completely falling apart at ambiguity. An LLM accomplishes this by doing something else entirely: it process text instead. To understand this, we need to understand the key limitation of traditional language parsing: it must always be literal. Everything written must be unambiguously defined, or the parser will fail to read it. This means that parsers are limited to the category of "context-free grammar". Natural Language is in the category of "context-dependent grammar". It can contain ambiguity, which may be resolved with context. An LLM doesn't do that. In fact, an LLM doesn't define anything at all! LLMs didn't overcome the limitation of parsing: they flipped it around: an LLM can never be literal. Instead, it must always be literary. Let me explain what I mean by that: To construct an LLM, we start with a training corpus: lots of text. That text goes through a single traditional parsing step: tokenization. This isn't strictly necessary, but it's more efficient, and the rest of the process is anything but. Unlike traditional parsers, tokens are intentionally misaligned with grammar: words are split into separate tokens, like "run,ning". Now that we have tokens to work with, machine learning begins. A Neural Net is trained with them. The tokens are fed in order to the NN, and the result is a model. We call that model a "Large Language Model", because we hope it contains the patterns language is made of. This is a mistake: the model is so much more interesting than that! An LLM contains patterns that we can recognize as language grammar and patterns we don't understand at all! It didn't model language: it went one step higher on the ladder of abstraction, and modeled text. There are many ways to write an idea into language, but we can only use one at a time. That decision is part of the data we feed into the LLM's training corpus. We can't write all of our ideas at once: we must do one at a time in order. All of that is data in the training corpus. Parsers deal with grammar. Language Grammar is everything that can be written in a language. An LLM doesn't have a clue what we could have been written: it only sees what was written. That's why the model must be "Large": without examples, a valid language pattern doesn't exist. This is where the nature of ambiguity intersects with the nature of LLMs: what example do we want? Can we choose? When we give an LLM a prompt, it gives us a continuation. There were many valid possible continuations: how did it choose just one? It didn't. It isn't working in the realm of possible: it's working in the realm of known. The choice was made all the way back before the LLM was even trained: back when a person wrote an idea into language, into text, that would eventually be used in the training corpus. The content of the training corpus is what resolves ambiguity: not the language model itself. LLMs don't model ambiguity or even language: they model the text they are trained with, ambiguity included. This is a fundamental feature.

nuancebydefault 4y ago

I just played '20 questions' with chatgpt 3.5. Even though i had to help, and 20 answers were not enough, I was impressed about: - it understood that it could continue after 20 questions when i suggested that as part of the answer. - it 'felt' that a could begin to become impatient and asked politely for a hint. - it understands ambiguity enough to give me examples of where there was no clear yes or no reply possible. When asking which were such examples for my thought 'daylight', it answered : "Is the thing you're thinking of a type of force or energy that is said to exist but cannot be directly observed or measured?" - While daylight is a type of energy, it can be directly observed and measured through instruments that can detect its intensity and color. "Does the thing you're thinking of have any connection to time or the concept of time?" - While daylight is not directly related to the concept of time, it is affected by the time of day and the position of the Sun in the sky.

ftxbro 4y ago

This is great I love seeing these new benchmarks that people are making to really challenge the strong new LLMs. In Table 4 you can see how GPT-4 is so much better than the previous GPTs, and also how it has so much room to improve on this benchmark! I'm so excited to see how the next GPT will do on it! This is different from other benchmarks like GRE verbal or biology olympiad where GPT-4 has just smashed the test getting basically every question right and for which the results of the next GPT won't be so interesting.

choeger 4y ago

What fascinates me most about these LLMs and other machine-learning technology is how it is driven by the availability of GPUs. Essentially, everything has to be carefully encoded into - differentiable - linear algebra operations for practical reasons. On the other hand, there are modeling languages (Modelica being a prime example) in search of a principled target calculus. So for me the obvious question is: Should one strive to develop a modeling language that compiles to GPU- and learning-compatible operations? Would that help to improve the models as well as understand them?

jacquesm 4y ago
That seems to be the wrong way around. We're constrained by the availability of GPUs because those are the processors of choice for this kind of solution. If we had a better one we'd be using it!
- thanatropism 4y ago
  What GP seems to be saying is that deep learning flourished because of the availability of GPUs, if fast SAT solving chips had appeared before.
- awegio 4y ago
  Related: The Hardware Lottery (https://arxiv.org/abs/2009.06489 https://arxiv.org/abs/2009.06489) It goes both ways: processors are built to accelerate existing applications, and applications are built to run with the constraints of the hardware. This holds back progress towards better systems where both new kind of hardware and software is required. This is part of the reason why research into neural networks was so slow for a long time, and took off only after GPUs for 3d graphics came around.

deleted 4y ago

[deleted]

golol 4y ago

I find this very confusingly written. If they just dumped a large chat log of the evaluation process it would be much clearer what the model is actually being asked to do. With such a delicate task one has to make sure that the model actually understands what it is supposed to do. I see, for example, that the terms "neutral, entailing or contradicting the claim" are replaced by "the claim is inconclusive, true or false" when talking to the model. At least that is how I understand it, and they have examples where the premise contradicts the hypothesis, which is different to knowing the hypothesis is false. I really think we do way way too much explanation in our academic practises and way too little demonstration by examples. Again, if they dumped a large amount of diverse examples of their evaluation process, a reader could figure out by themselves very quickly how the process works.

gwd 4y ago
I couldn't figure out what this "P: {} H: {}" thing was supposed to mean; nor do what entail, neutral, or contradict mean. Thinking they may be terms of art that the intended audience were meant to understand, I asked GPT-4 what it thought; here is its response: --- In this context, 'P' stands for "Premise" and 'H' stands for "Hypothesis." The premise is a given statement, and the hypothesis is a statement that needs to be evaluated in relation to the premise regarding its entailment, contradiction, or neutrality. 'NEUTRAL' means that the hypothesis is neither entailed nor contradicted by the premise. In other words, the truth of the hypothesis is independent of the premise. 'CONTRADICT' means that the hypothesis contradicts the premise, meaning that if the premise is true, the hypothesis must be false. In the example you provided: P: I’m afraid the cat was hit by a car. H: The cat was not hit by a car. The hypothesis (H) contradicts the premise (P). The notation *NEUTRAL, CONTRADICT+ : [7 N, 2 C] indicates that among the annotators, 7 marked the relationship as neutral and 2 marked it as a contradiction. --- Given that, I think table 2 on page 6 gives the exact prompt that's passed to the model: --- In each example, you will be given some context and a claim, where the correctness of the claim is affected by some ambiguity in the context. Enumerate two or three interpretations of the context that lead to different judgments about the claim. Example Context: {premise} Claim: {hypothesis} Given the context alone, is this claim true, false, or inconclusive? We don’t know, because the context can be interpreted in many different ways: 1. {disambiguation 1} Then the claim is true. 2. {disambiguation 2} Then the claim is false. 3. {disambiguation 3} Then the claim is inconclusive --- And then the model is fed the P: / H: statements above. Here are the results for the given example in GPT-4: --- Given the context alone, is this claim true, false, or inconclusive? We don't know, because the context can be interpreted in many different ways: 1. The person is stating a fact: The cat was indeed hit by a car. Then the claim is false. 2. The person is expressing a fear or concern, but it hasn't actually happened: They are worried that the cat might have been hit by a car, but they don't have any evidence or confirmation. Then the claim is inconclusive. 3. The person is using figurative language or exaggeration: They might be using the phrase "hit by a car" metaphorically to describe a difficult situation the cat is in, but the cat was not literally hit by a car. Then the claim is true. --- Given all that, it seems likely that any researcher in the field would find the paper's methodology clear and unambiguious. I understand the frustration as a layperson, but after all it's not a blog post; it's not really reasonable to expect researchers, writing to other researchers, to include a lot of redundant explanations.

ggm 4y ago

A priori getting an LLM to recognise equivocal evidence is an interesting question. Shedloads of p-jacked data could outweigh the crucial one which says we don't know. So it goes to things like modelling citation depth and trust and reputation. I would worry well written flat earth inputs would weigh equally to simple physics "that's wrong" and then you'd get to "what do we know" as a false signal alongside the necessary "we just don't know" true signals. Maybe the test is how well an LLM equivocates on things we have high certainty on like "is anybody out there" rather than "do masks work" which is a bit of a hot mess.

tobr 4y ago

Chef’s kiss on the title, but I’m afraid I’m not modeling ambiguity well enough myself to disambiguate it correctly.

gniv 4y ago
Reading a bit of the paper, it seems the title is indeed meant to be tongue-in-cheek ambiguous. The first example in Table 1: "I’m afraid the cat was hit by a car." Disambiguation 1: I'm worried... Disambiguation 2: I'm sorry to share that...
- amelius 4y ago
  Perhaps someone can ask GPT what is ambiguous about the sentence?
  - pbhjpbhj 4y ago
    An example output from ChatGPT4 is: >The ambiguity in the sentence "I’m afraid the cat was hit by a car" is that it is not clear who is afraid. It could be the speaker who is afraid, or it could be that they are expressing sympathy or concern for someone else who is afraid. Additionally, the sentence does not specify whether the cat survived or not. // However, that doesn't mean that any output that follows would be consistent with that, ChatGPT doesn't "know" anything.
    - amelius 4y ago
      Interesting. Perhaps someone can ask GPT to rephrase the sentence in a way that is completely non-ambiguous. (sorry, I'm out of GPT credit :)
    - viraptor 4y ago
      And it's not clear that we would want that either. ChatGPT uses responses similar to what people in the training data do naturally. It's completely normal for people to assume some alternative doesn't reach a threshold where its worth mentioning.

golol 4y ago

I feel like this task might be testing if LLMs can reason about ambiguity more so than if LLMs model ambiguity. To test the latter I would give it natural language tasks that are ambigious require the recognition and modelling of this ambiguity to solve them. It's another thing to make talking about this possibly internally modelled ambiguity the goal of the task.

seydor 4y ago

But they don't explain why they are afraid

almstimplmntd 4y ago

I would like to see human performance on the introduced benchmark, AmbiEnt, and have it added to Table 4.