5 ms·

>The machine has no semantic understanding of what the data represents. How do you define "semantic understanding" in a way that doesn't ultimately boil down t

>The machine has no semantic understanding of what the data represents. How do you define "semantic understanding" in a way that doesn't ultimately boil down to saying they don't have phenomenal consciousness? Any functional concept of semantic understanding is captured to some degree by LLMs. Typically when we attribute understanding to some entity, we recognize some substantial abilities in the entity in relation to that which is being understood. Specifically, the subject recognizes relevant entities and their relationships, various causal dependences, and so on. This ability goes beyond rote memorization, it has a counterfactual quality in that the subject can infer facts or descriptions in different but related cases beyond the subject's explicit knowledge. But LLMs excel at this. >feed it more tricks so that it doesn't fumble with basic questions like how many "R"s are in "strawberry" This failure mode has nothing to do with LLMs lacking intelligence and everything to do with how tokens are represented. They do not see individual characters, but sub-word chunks. It's like expecting a human to count the pixels in an image it sees on a computer screen. While not impossible, it's unnatural to how we process images and therefore error-prone.

TheOtherHobbes 1y ago
You don't need phenomenal consciousness. You need consistency. LLMs are not consistent. This is unarguable. They will produce a string of text that says they have solved a problem and/or done a thing when neither is true. And sometimes they will do it over and over, even when corrected. Your last paragraph admits this. Tokenisation on its own simply cannot represent reality accurately and reliably. It can be tweaked so that specific problems can appear solved, but true AI would be based on a reliable general strategy which solves entire classes of problems without needing this kind of tweaking. It's clear we're nowhere close to that.
- hackinthebochs 1y ago
  Consistency is a strange criteria seeing as humans aren't very consistent either. Intelligent beings can make mistakes.
  - danaris 1y ago
    This is a common category of error people commit when talking about LLMs. "True, LLMs can't do X, but a lot of people don't do X well either!" The problem is, when you say humans have trouble with X, what you mean is that human brains are fully capable of X, but sometimes they do, indeed, make mistakes. Or that some humans haven't trained their faculties for X very well, or whatever. But LLMs are fundamentally, completely, incapable of X. It is not something that can be a result of their processes. These things are not comparable. So, to your specific point: When an LLM is inconsistent, it is because it is, at its root, a statistical engine generating plausible next tokens, with no semantic understanding of the underlying data. When a human is inconsistent, it is because they got distracted, didn't learn enough about this particular subject, or otherwise made a mistake that they can, if their attention is drawn to it, recognize and correct. LLMs cannot. They can only be told they made a mistake, which prompts them to try again (because that's the pattern that has been trained into them for what happens when told they made a mistake). But their next try won't have any better odds of being correct than their previous one.
    - hackinthebochs 1y ago
      >But LLMs are fundamentally, completely, incapable of X. It is not something that can be a result of their processes. This is the very point of contention. You don't get to just assume it. > it is because it is, at its root, a statistical engine generating plausible next tokens, with no semantic understanding of the underlying data. Another highly contentious point you are just outright assuming. LLMs are modelling the world, not just "predicting the next token". Some examples here[1][2][3]. Anyone claiming otherwise at this point is not arguing in good faith. It's interesting how the people with the strongest opinions about LLMs don't seem to understand them. [1] https://arxiv.org/abs/2405.15943 https://arxiv.org/abs/2405.15943 [2] https://x.com/OwainEvans_UK/status/1894436637054214509 https://x.com/OwainEvans_UK/status/1894436637054214509 [3] https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language...
      - danaris 1y ago

TheOtherHobbes 1y ago

You don't need phenomenal consciousness. You need consistency. LLMs are not consistent. This is unarguable. They will produce a string of text that says they have solved a problem and/or done a thing when neither is true. And sometimes they will do it over and over, even when corrected. Your last paragraph admits this. Tokenisation on its own simply cannot represent reality accurately and reliably. It can be tweaked so that specific problems can appear solved, but true AI would be based on a reliable general strategy which solves entire classes of problems without needing this kind of tweaking. It's clear we're nowhere close to that.

hackinthebochs 1y ago
Consistency is a strange criteria seeing as humans aren't very consistent either. Intelligent beings can make mistakes.
- danaris 1y ago
  This is a common category of error people commit when talking about LLMs. "True, LLMs can't do X, but a lot of people don't do X well either!" The problem is, when you say humans have trouble with X, what you mean is that human brains are fully capable of X, but sometimes they do, indeed, make mistakes. Or that some humans haven't trained their faculties for X very well, or whatever. But LLMs are fundamentally, completely, incapable of X. It is not something that can be a result of their processes. These things are not comparable. So, to your specific point: When an LLM is inconsistent, it is because it is, at its root, a statistical engine generating plausible next tokens, with no semantic understanding of the underlying data. When a human is inconsistent, it is because they got distracted, didn't learn enough about this particular subject, or otherwise made a mistake that they can, if their attention is drawn to it, recognize and correct. LLMs cannot. They can only be told they made a mistake, which prompts them to try again (because that's the pattern that has been trained into them for what happens when told they made a mistake). But their next try won't have any better odds of being correct than their previous one.
  - hackinthebochs 1y ago
    >But LLMs are fundamentally, completely, incapable of X. It is not something that can be a result of their processes. This is the very point of contention. You don't get to just assume it. > it is because it is, at its root, a statistical engine generating plausible next tokens, with no semantic understanding of the underlying data. Another highly contentious point you are just outright assuming. LLMs are modelling the world, not just "predicting the next token". Some examples here[1][2][3]. Anyone claiming otherwise at this point is not arguing in good faith. It's interesting how the people with the strongest opinions about LLMs don't seem to understand them. [1] https://arxiv.org/abs/2405.15943 https://arxiv.org/abs/2405.15943 [2] https://x.com/OwainEvans_UK/status/1894436637054214509 https://x.com/OwainEvans_UK/status/1894436637054214509 [3] https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language...
    - danaris 1y ago