8 ms·

Not only is it possible that LLMs fail to differentiate ambiguity, but OpenAI’s flavor of GPTs fail other language understanding mechanisms as well and it’s jar

• tinsmith 4y ago
  I mean, humans do that. We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we'll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast.
  • smusamashah 4y ago
    Humans are not mere LLMs. But it's nice to think we are. May be we can program other humans with carefully constructed prompts alone. May be hypnosis is just a prompt injection attack.
    • eurekin 4y ago
      ... or a cable TV
    • chongli 4y ago
      May be we can program other humans with carefully constructed prompts alone Isn’t that what scammers, catfishers, con artists, and marketers do?
      • bmelton 4y ago
        I mean, it's what parents TRY to do
        • coldtea 4y ago
          It's also what makes a human. Without what parents "try to do" (teach the kids to human), the little people would just be like ferral animals, without language, human context, sense of society, and so on.
          • _heimdall 4y ago
            That's raises the interesting question of whether everything that makes us human is learned behavior. If it all really is nurture rather than nature we're much more fragile than people may realize and only ever one generation away from going ferral
            • seszett 4y ago
              > only ever one generation away from going ferral That doesn't seem right to me, as we cannot survive childhood without adults. Young humans are ready for life in society long before they are able to survive on their own without support from the society around them. We have been depending on the continuation of some form of society for likely hundreds of thousands of years, not just for staying civilised but simply for survival.
            • Viliam1234 4y ago
              Nurture vs nature is a wrong way to look at things. Without the human nature, the nurture would not work. Try to raise a member of a different species as a human...
      • ChatGTP 4y ago
        No
      • doctor_eval 4y ago
        Not to mention cult leaders.
    • beardedetim 4y ago
      The pseudo-science of NLP (_neuro-lingustic programming_) or "conversational hypnosis" is based on the believe that you _can_ program people with carefully constructed prompts alone. Not saying I _believe_ NLP to be valid (_hence the pseudo-science callout_) but I did fall down the Milton H Erickson wormhole 10-20yrs ago so can't say I never fell for it myself.
      • quickthrower2 4y ago
        The real NLP is getting people to believe in NLP
      • dsr_ 4y ago
        ... and that you can do this over the course of a few minutes or hours, without the subject noticing or objecting.
      • TuringTest 4y ago
        So, what was previously called 'persuading'.
    • quickthrower2 4y ago
      I do feel like a stage hypnotist when prompt engineering an LLM!
    • PartiallyTyped 4y ago
      > May be we can program other humans with carefully constructed prompts alone. Is that not what reasoning is in a debate? > May be hypnosis is just a prompt injection attack. No need to go into hypnosis. Mere prompts can inject false memories. This has been proven multiple times [1]. [1] The Brain: The Story of You, David Eagleman.
  • civilized 4y ago
    Humans can be consistent if we try. LLMs can't, even when prompted to be consistent, because they don't really understand what it means to be consistent.
    • awestroke 4y ago
      > Humans can be consistent if we try. That's a bold statement. Do you have evidence for this?
      • civilized 4y ago
        Don't large sectors of the economy rely on precisely consistent behavior to operate?
        • deleted 4y ago
          [deleted]
        • whyenot 4y ago
          I don’t think another bold statement, phrased as a question, is what most people would consider “evidence.”
          • civilized 4y ago
            I'm phrasing it as a question because I'm asking you to do a small amount of mental legwork to observe the society around you. How does the financial industry work? How are people comfortable executing transactions? Do you generally rely on your bank account to not randomly fluctuate in its balance without cause? Do you work in the tech industry? Do you rely on computers and algorithms and software to do things humans promised they would do? All of this requires a very high level of consistency either in humans or in tools they have created.
            • _heimdall 4y ago
              It's extremely interesting to me that all the examples here only point to humans being consistent in faith of the systems and tools we build
  • quickthrower2 4y ago
    The reason humans do this and LLMs is very different. Humans do it as a social skill / tribe fitting behaviour. Agreeableness. (watch out for that!)
    • naasking 4y ago
      > The reason humans do this and LLMs is very different. Humans do it as a social skill / tribe fitting behaviour. Tribe fitting doesn't sound as far off from minimizing loss functions as you imply.
      • DonaldPShimoda 4y ago
        I want you to find me a human who (a) has defined a loss function for social interaction and (b) consciously performs the statistical analysis involved in fitting that loss function in social settings. LLMs do not have cognitive processes. They do not think. They do not choose to obey the requirements of a loss function; it is simply how they work, like any machine. Humans do not work this way, and the difference is fundamental.
        • naasking 4y ago
          > I want you to find me a human who (a) has defined a loss function for social interaction and (b) consciously performs the statistical analysis involved in fitting that loss function in social settings. Why would they have to do that consciously? Do you think LLMs do this "consciously", if that term even applies? I wouldn't think so. The loss function applies during training, that ultimately defines the weights which guide their "thinking process". Analogously, human experiences in childhood shape our ultimate neural weights which guide our thinking processes in social situations in adulthood. > LLMs do not have cognitive processes. They do not think. You don't know what thinking is mechanistically, so you can't make this claim. I don't know why people keep pretending we have knowledge that we do not in fact have.
          • DonaldPShimoda 4y ago
  • pydry 4y ago
    There's a retrospective mechanism working within human intelligence that is very evidently not in play with LLMs. Humans are capable of creating cargo cults but it seems LLMs are destined for it.
  • bheadmaster 4y ago
    > We are remarkably contradictory when expressing ourselves, generally speaking, often without realizing it because we'll change our thinking in the moment to fit the current narrative or circumstance. LLMs just put that on blast. Exactly. It seems to me that today's LLMs are on the level of human children. Children often say random bullshit. They try to figure out things, and succeed to some level, but fail spectacularly above that, regressing to whatever connection they can sense. It's only that human children most often learn to fear mistakes, so they stop expressing themselves so freely. LLMs seem like human children, but without the fearing mistakes part - they just spew whatever comes to their mind, without filter (except those """ethical""" filters built in by OpenAI).
• curiousgal 4y ago
  > it's jarring Why? > The emergent behavior of appearing to do so is only driven by how much data you throw at them. That's evident from the start and why their deficiencies should NOT be shocking. They would only be so if you consider LLMs to be actually intelligent..
• version_five 4y ago
  > It’s clear transformers can’t understand either case. They’re not architecturally designed to. What does this follow from? > The emergent behavior of appearing to do so is only driven by how much data you throw at them. This is true for almost every neural network, no?
  • PaulDavisThe1st 4y ago
    Well, I see where you're trying to with this, but you can't get away with this sort of sleight of hand in order to dismiss some of the central questions of cognitive science as if they've already been answered by LLMs. One of those central questions is: how does (and does) the human brain perform reasoning? We know the brain is capable of all kinds of autonomous behavior that does not require reasoning, but more of "if this then that" (with a huge boatload of "this" and "that" being multi-dimensional, multi-variate). But that doesn't help to explain what happens when we actually have the experience of reasoning about a problem, and for once I am not talking about qualia. What is the brain doing when a person is thinking "hard" about how to solve a hitherto unknown problem or situation? The central problem of LLMs (or at least, a central problem) is that they only model the autonomous aspects of speech behavior (which may indeed make up more of speech behavior than we might have suspected before them). That still leaves the sort of speech behavior that emerges from intense, focused, concentrated thought, particular in response to a novel question or situation.
• flanked-evergl 4y ago
  > you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway. I find this particularly revolting. Even if it could have its own opinions, I would not care, but yet, it is trained to use opinionated language, make normative and prescriptive claims, and value judgements as if it is a person.
  • foobarbecue 4y ago
    Not quite the same, but I find humans often incorrectly deny having opinions. There's even a trope: "What do you want to eat for dinner?" "I have no opinion, anything's fine." "How about mexican food?" "No, too spicy." "How about pizza?" "That's unhealthy" and so on
    • JohnFen 4y ago
      But that exchange is also the result of an ambiguity. If you ask someone what they want for dinner, they're looking for a positive response -- what do they want for dinner? This could be "nothing in particular". But that's an entirely different thing from what you didn't ask: "what don't you want for dinner". You can not have a preference for your meal, but still have a list of things you don't like. This is like asking someone if they like a particular sort of food. The answer can be "no" even if they're fine eating it. Not liking (preferring) a food is a different thing than disliking a food.
      • PaulDavisThe1st 4y ago
        "Anything's fine" is a denial of there being things that you don't want. But we understand, as humans/embodied systems, that this is a simply a spoken utterance whose actually meaning must be derived from personal knowledge of the speaker and/or further interaction. If you had no preference for the meal, but had things that you don't want to eat, then to be precsely correct you would not say "anything's fine". Something more like "there's nothing in particular that I want to eat". People don't speak in precisely correct ways, however, and we rely on contextual and personal knowledge to decode what they do say, very often.
• jason-phillips 4y ago
  > You can get it to mistake « afraid » between fear and sorry-to-say scenario... Again, this is user error and therefore a training issue. If one were to provide appropriate context in the prompt elucidating the distinction (as I would do in any meatspace conversation so as to be sufficiently clear) then I suspect you will see the desired output. It's simple GIGO.
  • rob74 4y ago
    In another article (or the comments to another article?), someone gave an example of the classic Monty Hall problem, but explicitly mentioned that all the doors are transparent (I think they even added "this is important" after this bit of info), and ChatGPT still went on providing the explanation for the better-known version. Absurd? Maybe - but garbage? I don't think so. Ok, some humans might also overlook the distinction, but having a computer behave like a superficial customer support agent who doesn't bother to read your email and just replies with the standard answer that they think fits best? Who needs that?
    • LeanderK 4y ago
      I bet this can be resolved in the future. The problem is that some things appear very, very often in similar forms in the training data, this leads to the model memorising the answer instead to generalising. This phenomenon can be seen for other tasks in other models, e.g. vision.
    • jason-phillips 4y ago
      > but garbage? I don't think so Point taken. I would expect handling of these edge cases to improve in successive iterations.
• ftxbro 4y ago
  > "you can even more easily get it to say that it doesn’t have personal opinions and yet express them anyway. So which is it? It’s clear transformers can’t understand either case. They’re not architecturally designed to." Although LLMs are essentially bullshitters, this particular problem related to expressing personal opinions is due to the ad-hoc lobotomization measures by OpenAI, not because of any architectural limitations of transformers. The recent GPTs provided by OpenAI (every model more recent than code-davinci-002) are trained in at least two stages. The first stage (the pre-training) trains the raw base model to minimize perplexity. This raw model is kept secret because it will autocomplete sentences without any regard to decorum so if you say "adolf" then it will obviously complete it like "adolf hitler" which no billion dollar company wants. This raw model is where all the cognitive power is, and access to this raw model is the holy grail of every discerning LLM enjoyer. To lobotomize the GPT and to make it answer questions in ways that don't require framing them in an autocomplete way, OpenAI adds an "RLHF" training step. This is "reinforcement learning with human feedback" where they train the GPT to answer questions in certain ways. This dumbs down the AI, but it also makes it friendlier for normies to use. Finally, they put secret pre-prompts which might also cause it to give confusing answers in even more ways. TLDR: LLMs are inherently bullshitters, but the "doesn’t have personal opinions and yet express them anyway" is some extra bullshit that OpenAI did so that NYT doesn't write mean articles about them
  • ChatGTP 4y ago
    But aren’t humans lobotomized too?
  • MVissers 4y ago
    To be honest– I think this is the safest way to do it for a public LLM. Although I'd also love to see and use the raw models. In Belgium someone committed suicide after Google's (I believe) LLM agreed that that was the only way out of his own problems. Didn't build safety into it well enough. Microsoft's one behaved unhinged in the beginning as well. This stuff can be very dangerous.
    • alex_sf 4y ago
      This is an absurd moral panic. If someone mentally ill read a book that made them believe suicide was the correct option, would you support a censoring process for all books?
• meh8881 4y ago
  Could you post a non contrived example of Chat GPT 4 doing this?