6 ms·

> understand the work LLMs don't understand. It's mind-boggling to me that large parts of the tech industry think that. Don't ascribe to them what they don't

by yusina 1y ago

> understand the work LLMs don't understand. It's mind-boggling to me that large parts of the tech industry think that. Don't ascribe to them what they don't have. They are fantastic at faking understanding. Don't get me wrong, for many tasks, that's good enough. But there is a fundamental limit to what all this can do. Don't get fooled into believing there isn't.

neom 1y ago
What is the limit my system will reach?
rzz3 1y ago
Thats an interesting word to pick on. Understanding still means something here in a relative sense.
zenburnmyface 1y ago
meh. I feel this is just a linguistic shortcut, similar to how _trained_ biologists can talk about a species or organism evolving some trait. Of course the organism isn't _really_ evolving with any goal in mind, but that's clear to the speaker and audience. Whether or not LLMs understand (very unlikely), it's clear what we mean by an LLM "understanding": has the context + prior training to make reasonable predictions. But no one wants to write that each time.
- yusina 1y ago
  That's an interesting take and in fact one I could get behind. But I'm afraid that most folks using the term mean it more literally than you describe.
  - philistine 1y ago
    Exactly. The whole point of all the LLM companies is to get grandma to use it. If you say understand about a technology with the desired appeal of Facebook, then you’re talking to everyone and words matter extra hard.
motorest 1y ago
> LLMs don't understand. It's mind-boggling to me that large parts of the tech industry think that. I think you might be tied to a definition of "understanding" that doesn't really apply. If you prompt a LLM with ambiguous instructions, it requests you to clarify (i.e., extend prompt to provide more context) and once you do the LLM outputs something that exactly meets the goals of the initial prompt, does it count as understanding? If it walks like a duck and quacks like a duck, it's a duck,or something so close to a duck that we'd be better off calling it that.
- dingnuts 1y ago
  [flagged]
  - Uehreka 1y ago
    > NO. YES! Sometimes. You’ll often hear the term “zero-shot generation”, meaning creating something new given zero examples, this is something many modern models are capable of.
  - childintime 1y ago
    In my experience it generally has a very good understanding and does generate the relevant test cases. Then again I don't give it a grammar, I just let it generalize from examples. In my defense I've tried out some very unconventional languages. Grammars are an attempt at describing a language. A broken attempt if you ask me. Humans also don't like them.
    - yusina 1y ago
      For natural language you are right. The language came first, the grammar was retrofitted to try to find structure. For formal languages, which programming languages (and related ones like query languages, markup languages, etc) are an instance of, the grammar defines the language. It come first, examples second. Historically, computers were very good at formal languages. With LLMs we are entering a new age where machines are becoming terrible at something they once excelled at. Have you lately tried asking Google whether it's 2025? The very first data keeping machines (clocks) were also pretty unreliable at that. Full circle I guess.
vasco 1y ago
Asking a short question but in a serious way: so what?
- yusina 1y ago
  You are asking why it is meaningful to use terms for what they mean instead of making up things? Well, I prefer it that way, but the spirit of "AI" seems to go in another direction, and the leadership of US government also does, so maybe times are just changing.
hayst4ck 1y ago
Nearly every argument like this has the same fatal flaw, and it's generally not the critique of the AI, but the critique reflected back on to humans. Humans also don't understand and are frequently faking understanding, which for many tasks is good enough. There are fundamental limits to what humans can do. The AI of a few months ago before OpenAI's sycophancy was quite impressive, less so now which means it is being artificially stunted so more can be charged later. It means privately it is much better than what is public. I can't say it "understands," but I can say it outclasses many many humans. There are already numbers of tasks based around understanding where I would already choose an LLM over a human. It's worth looking at bloom's taxonomy (https://en.wikipedia.org/wiki/Bloom%27s_taxonomy https://en.wikipedia.org/wiki/Bloom%27s_taxonomy): In the 2001 revised edition of Bloom's taxonomy, the levels were renamed and reordered: Remember, Understand, Apply, Analyze, Evaluate, and Create. In my opinion it is at least human competitive for everything but create. I used to be very bearish on AI, but if you haven't had a "wow" moment when using one, then I don't think you've tried to explore what it can do or tested it's limits with your own special expertise/domain knowledge, or if you have then I'm not sure we're using the same LLMs. Then compare that experience to normal people, not your peer groups. Compare an LLM to people into astrology, crystal healing, or homeopathy and ask which has more "understanding."
- yusina 1y ago
  Um, moving the goal post? The claim was LLMs understand things. The counter was, nope, they don't. They can fake it well though. Your argument now is, well humans also often fake it. Kinda implying that it means it's ok to claim that LLMs have understanding? They may outclass people in a bunch of things. That's great! My pocket calculator 20 years also did, and it's also great. Neither understands what they are doing though.
  - neom 1y ago
    It's fun to talk about, but personally he whole "understanding" debate is a red herring, imo what we actually care about when we talk about intelligence is the capacity and depth of: second order thinking, regardless of the underlying mechanism. I think personally key question isn't "do LLMs understand?" but, "can LLMs engage in second order thinking?" The answer seems to be yes - they can reason about reasoning, plan their approaches, critique their own outputs, and adapt their strategies, o1 has shown us that with RL and reasoning tokens you can include it in a single system, but our brains have multiple systems we can control and that can be combined in multiple ways at any given moment: emotions, feelings, thoughts combined into user space, 3 core systems input, memory, output. The nuances is in the fact that various reasons (nature + nurture), various humans appear to have varying levels of meta control over the multiple reasoning systems.
bobxmax 1y ago
How do you know?
- yusina 1y ago
  Extraordinary claim requires extraordinary proof. I don't know, but I'm also not the one claiming something. (Besides, we know what LLMs do, and none of those things indicate understanding. Just statistics.)
  - shawabawa3 1y ago
    You can create a new game with new rules never seen before You can explain this to an LLM The LLM can then play the game following the rules How can you say it hasn't understood the game?
    - yusina 1y ago
      The LLM is only capable of doing so if it has encountered something similar before as part of its training. Claiming anything else requires a proof.
      - neom 1y ago
        the extraordinary claim would be that LLMs can only do things they've seen before exactly, given the compositional and emergent capabilities we observe. The evidence suggests they can generalize beyond their training in meaningful ways, even if imperfectly...if a human came out living but with a brain that had zero electrical activity, that would be extraordinary, we normally come out with a baseline of pre-programming. I sometimes think this debate happens because humans don't want to admit we're nothing more than LLMs programmed by nature and nurture, human seem to want to be especially special. https://arxiv.org/abs/2206.07682 https://arxiv.org/abs/2206.07682 https://towardsdatascience.com/enhanced-large-language-models-as-reasoning-engines-582bff782113/ https://towardsdatascience.com/enhanced-large-language-model... https://arxiv.org/abs/2308.00304 https://arxiv.org/abs/2308.00304 (and if MoRA is moving the goal posts, fine: RL/RT)
GaggiX 1y ago
They understand tho, it's different than how it's done in our brain but they solve task that would be impossible to do without understanding. I would even say that they can now reason through problems thanks to powerful reasoning models like Gemini 2.5 Pro and o3.
GoatInGrey 1y ago
I don't believe the user meant "understand" in the classical biological and philosophical sense, or were otherwise attempting to anthropomorphize the systems. They were speaking from the practical experience of "this thing takes a somewhat ambiguous input with unique constraints and implements the ask more-or-less as intended".
squidbeak 1y ago
They understand. Anything able to reason about any arbitrary request and form a plan tailored to that request understands well enough to qualify for the verb. The mechanism behind it may feel hollow or fake. But if its responses reliably show understanding, the LLM understands - by any ordinary measure.
- intended 1y ago
  Rote learning is a term that exists which specifically punctures this output oriented measurement of understanding.
holoduke 1y ago
Definition of understanding is based on connecting relations. If there is one thing a llm can do its connecting relations. So I am not sure why you say llms are not understanding.

by yusina 1y ago

neom 1y ago

What is the limit my system will reach?

rzz3 1y ago

Thats an interesting word to pick on. Understanding still means something here in a relative sense.

zenburnmyface 1y ago

meh. I feel this is just a linguistic shortcut, similar to how _trained_ biologists can talk about a species or organism evolving some trait. Of course the organism isn't _really_ evolving with any goal in mind, but that's clear to the speaker and audience. Whether or not LLMs understand (very unlikely), it's clear what we mean by an LLM "understanding": has the context + prior training to make reasonable predictions. But no one wants to write that each time.

yusina 1y ago
That's an interesting take and in fact one I could get behind. But I'm afraid that most folks using the term mean it more literally than you describe.
- philistine 1y ago
  Exactly. The whole point of all the LLM companies is to get grandma to use it. If you say understand about a technology with the desired appeal of Facebook, then you’re talking to everyone and words matter extra hard.

motorest 1y ago

> LLMs don't understand. It's mind-boggling to me that large parts of the tech industry think that. I think you might be tied to a definition of "understanding" that doesn't really apply. If you prompt a LLM with ambiguous instructions, it requests you to clarify (i.e., extend prompt to provide more context) and once you do the LLM outputs something that exactly meets the goals of the initial prompt, does it count as understanding? If it walks like a duck and quacks like a duck, it's a duck,or something so close to a duck that we'd be better off calling it that.

dingnuts 1y ago
[flagged]
- Uehreka 1y ago
  > NO. YES! Sometimes. You’ll often hear the term “zero-shot generation”, meaning creating something new given zero examples, this is something many modern models are capable of.
- childintime 1y ago
  In my experience it generally has a very good understanding and does generate the relevant test cases. Then again I don't give it a grammar, I just let it generalize from examples. In my defense I've tried out some very unconventional languages. Grammars are an attempt at describing a language. A broken attempt if you ask me. Humans also don't like them.
  - yusina 1y ago
    For natural language you are right. The language came first, the grammar was retrofitted to try to find structure. For formal languages, which programming languages (and related ones like query languages, markup languages, etc) are an instance of, the grammar defines the language. It come first, examples second. Historically, computers were very good at formal languages. With LLMs we are entering a new age where machines are becoming terrible at something they once excelled at. Have you lately tried asking Google whether it's 2025? The very first data keeping machines (clocks) were also pretty unreliable at that. Full circle I guess.

vasco 1y ago

Asking a short question but in a serious way: so what?

yusina 1y ago
You are asking why it is meaningful to use terms for what they mean instead of making up things? Well, I prefer it that way, but the spirit of "AI" seems to go in another direction, and the leadership of US government also does, so maybe times are just changing.

hayst4ck 1y ago

Nearly every argument like this has the same fatal flaw, and it's generally not the critique of the AI, but the critique reflected back on to humans. Humans also don't understand and are frequently faking understanding, which for many tasks is good enough. There are fundamental limits to what humans can do. The AI of a few months ago before OpenAI's sycophancy was quite impressive, less so now which means it is being artificially stunted so more can be charged later. It means privately it is much better than what is public. I can't say it "understands," but I can say it outclasses many many humans. There are already numbers of tasks based around understanding where I would already choose an LLM over a human. It's worth looking at bloom's taxonomy (https://en.wikipedia.org/wiki/Bloom%27s_taxonomy https://en.wikipedia.org/wiki/Bloom%27s_taxonomy): In the 2001 revised edition of Bloom's taxonomy, the levels were renamed and reordered: Remember, Understand, Apply, Analyze, Evaluate, and Create. In my opinion it is at least human competitive for everything but create. I used to be very bearish on AI, but if you haven't had a "wow" moment when using one, then I don't think you've tried to explore what it can do or tested it's limits with your own special expertise/domain knowledge, or if you have then I'm not sure we're using the same LLMs. Then compare that experience to normal people, not your peer groups. Compare an LLM to people into astrology, crystal healing, or homeopathy and ask which has more "understanding."

yusina 1y ago
Um, moving the goal post? The claim was LLMs understand things. The counter was, nope, they don't. They can fake it well though. Your argument now is, well humans also often fake it. Kinda implying that it means it's ok to claim that LLMs have understanding? They may outclass people in a bunch of things. That's great! My pocket calculator 20 years also did, and it's also great. Neither understands what they are doing though.
- neom 1y ago
  It's fun to talk about, but personally he whole "understanding" debate is a red herring, imo what we actually care about when we talk about intelligence is the capacity and depth of: second order thinking, regardless of the underlying mechanism. I think personally key question isn't "do LLMs understand?" but, "can LLMs engage in second order thinking?" The answer seems to be yes - they can reason about reasoning, plan their approaches, critique their own outputs, and adapt their strategies, o1 has shown us that with RL and reasoning tokens you can include it in a single system, but our brains have multiple systems we can control and that can be combined in multiple ways at any given moment: emotions, feelings, thoughts combined into user space, 3 core systems input, memory, output. The nuances is in the fact that various reasons (nature + nurture), various humans appear to have varying levels of meta control over the multiple reasoning systems.

bobxmax 1y ago

How do you know?

yusina 1y ago
Extraordinary claim requires extraordinary proof. I don't know, but I'm also not the one claiming something. (Besides, we know what LLMs do, and none of those things indicate understanding. Just statistics.)
- shawabawa3 1y ago
  You can create a new game with new rules never seen before You can explain this to an LLM The LLM can then play the game following the rules How can you say it hasn't understood the game?
  - yusina 1y ago
    The LLM is only capable of doing so if it has encountered something similar before as part of its training. Claiming anything else requires a proof.
    - neom 1y ago
      the extraordinary claim would be that LLMs can only do things they've seen before exactly, given the compositional and emergent capabilities we observe. The evidence suggests they can generalize beyond their training in meaningful ways, even if imperfectly...if a human came out living but with a brain that had zero electrical activity, that would be extraordinary, we normally come out with a baseline of pre-programming. I sometimes think this debate happens because humans don't want to admit we're nothing more than LLMs programmed by nature and nurture, human seem to want to be especially special. https://arxiv.org/abs/2206.07682 https://arxiv.org/abs/2206.07682 https://towardsdatascience.com/enhanced-large-language-models-as-reasoning-engines-582bff782113/ https://towardsdatascience.com/enhanced-large-language-model... https://arxiv.org/abs/2308.00304 https://arxiv.org/abs/2308.00304 (and if MoRA is moving the goal posts, fine: RL/RT)

GaggiX 1y ago

They understand tho, it's different than how it's done in our brain but they solve task that would be impossible to do without understanding. I would even say that they can now reason through problems thanks to powerful reasoning models like Gemini 2.5 Pro and o3.

GoatInGrey 1y ago

I don't believe the user meant "understand" in the classical biological and philosophical sense, or were otherwise attempting to anthropomorphize the systems. They were speaking from the practical experience of "this thing takes a somewhat ambiguous input with unique constraints and implements the ask more-or-less as intended".

squidbeak 1y ago

They understand. Anything able to reason about any arbitrary request and form a plan tailored to that request understands well enough to qualify for the verb. The mechanism behind it may feel hollow or fake. But if its responses reliably show understanding, the LLM understands - by any ordinary measure.

intended 1y ago
Rote learning is a term that exists which specifically punctures this output oriented measurement of understanding.

holoduke 1y ago

Definition of understanding is based on connecting relations. If there is one thing a llm can do its connecting relations. So I am not sure why you say llms are not understanding.