5 ms·

LLMs have access to the space of collective semantic understanding. I don't understand why people expect cognitive faculties that are clearly extra-semantic to

LLMs have access to the space of collective semantic understanding. I don't understand why people expect cognitive faculties that are clearly extra-semantic to just fall out of them eventually. The reason they sometimes appear to reason is because there's a lot of reasoning in the corpus of human text activity. But that's just a semantic artifact of a non-semantic process. Human cognition is much more than just our ability to string sentences together.

matt-attack 2y ago
I don’t know if you’re correct. I don’t think you know that our brains are that different? We too need to train ourselves on massive amounts of data. I feel like the kids of reasoning and understanding I’ve seen ChatGPT do are soooo far beyond something like just processing language.
- Minor49er 2y ago
  What reasoning have you seen coming from ChatGPT?
- thuuuomas 2y ago
  We do represent much of our cognition in language. Sometime I feel like LLMs might be “dancing skeletons” - pulleys & wire giving motion to the bones of cognition.
  - andai 2y ago
    But why stop there? All matter and all life is just increasingly fancy machines. https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie Editor's note: I do not promote such a worldview -- my intention is precisely the opposite.
- sgt101 2y ago
  Our brains have effects that proceeded language. Look at lions for an example. We are much more (and a little less) than lions in terms of mind.
- andai 2y ago
  When I talk to 8B models, it's often painfully clear that they are operating mostly (entirely?) on the level of language. They often say things that make no sense except from a "word association" perspective. With bigger (400B models) that's not so clear. It would be silly to say that a fruit fly has the same thoughts as me, only a million times smaller quantitatively. I imagine the same thing is true (genuine qualitative leaps) in the 8B -> 400B direction.
Torkel 2y ago
Multimodal models are not limited by semantic understanding though, right? They are given photos, video, audio.
- elzbardico 2y ago
  For them, it is all a stream of tokens, roughly speaking.
mbil 2y ago
I might expect some extra-semantic cognitive faculties to emerge from LLMs, or at least be approximated by LLMs. Let me try to explain why. One example of extra-semantic ability is spatial reasoning. I can point to a spot on the ground and my dog will walk over to it — he’s probably not using semantic processing to talk through his relationship with the ground, the distance of each pace, his velocity, etc. But could a robotic dog powered by an LLM use a linguistic or symbolic representation of spacial concepts and actions to translate semantic reasoning into spacial reasoning? Imagine sensors with a measurement to language translation layer (“kitchen is five feet in front of you”), and actuators that can be triggered with language (“move forward two feet”). It seems conceivable that a detailed enough representation of the world, expressive enough controls, and a powerful enough LLM could result in something that is akin to spacial reasoning (an extra-semantic process), while under the hood it’s “just” semantic understanding.
- defrost 2y ago
  Spatial reasoning is more akin to visualising a 3D "odd shaped" fuel tank from 2D schematics and being able to mentally rotate that shape to estimate where a fluid line would be at various angles. This is distinct from stringing together treasure map instructions in an chain.
  - throwuxiytayq 2y ago
    Isn’t spatial navigation a bit like graph walking, though? Also, AFAIK blind people describe it completely differently, and they’re generally confused by the whole concept of 3D perspective and objects getting visually smaller over distance, and so on. Brains don’t work the same for everyone in our species, and I wouldn’t presume to know the full internal representation just based on qualia.
  - 082349872349872 2y ago
    I'm always impressed by the "straightedge-and-compass"-flavoured techniques drafters of old used to rotate views of odd 3D shapes from pairs of 2D schematics, in the centuries before CAD software. https://en.wikipedia.org/wiki/Descriptive_geometry#Finding_the_shortest_connector_between_two_given_skew_lines_PR_and_SU https://en.wikipedia.org/wiki/Descriptive_geometry#Finding_t... https://en.wikipedia.org/wiki/Gaspard_Monge#1789_and_after https://en.wikipedia.org/wiki/Gaspard_Monge#1789_and_after
ninetyninenine 2y ago
Do you have any evidence that human cognition (for speaking) is more than just an ability to string sentences together? Do you have any evidence that LLMs don't reason at all? A perfect machine designed to only string sentences together as perfect responses with no reasoning built it IS Indistinguishable from a machine that only builds sentences from pure reasoning. Either way nobody understands what's going on in the human brain and nobody understands why LLMs work. You don't know. You're just stating a belief.
- slashdave 2y ago
  There is evidence that many animals have reasoning abilities. Few can talk.
  - ninetyninenine 2y ago
    There's also counter evidence that animals lack reasoning abilities. Ever see a gorilla attack it's own reflection or a dog chase it's own tail? Contradictory evidence displaying the ability to reason and the lack of the ability to reason is what is displayed by animals. Figures that the LLM displays the same contradictory evidence. But none of this evidence proves anything definitively. Much like human cognition, The LLM is a machine that we built but don't understand. No definitive statement can be made about it.
    - BriggyDwiggs42 2y ago
      You just argued that, because some animals display errors in reasoning, we can call into doubt the claim that any animal reasons. This does not follow. The reflection test, for example, is passed by some animals; we can test it by putting a red dot on their face and seeing if they mess with it. Either way, it’s not even a test of general reasoning abilities. I think you’re giving animals far too little credit.
      - sibeliuss 2y ago
        Not to mention all of the developments in mirror testing to account for differences in perception. What they're finding is that self-recognition is more common than assumed.
ekianjo 2y ago
> Human cognition is much more than just our ability to string sentences together animals without similar language capabilities dont seem to be too strong at reasoning. it could well be that language and reasoning are heavily linked together

matt-attack 2y ago

I don’t know if you’re correct. I don’t think you know that our brains are that different? We too need to train ourselves on massive amounts of data. I feel like the kids of reasoning and understanding I’ve seen ChatGPT do are soooo far beyond something like just processing language.

Minor49er 2y ago
What reasoning have you seen coming from ChatGPT?
thuuuomas 2y ago
We do represent much of our cognition in language. Sometime I feel like LLMs might be “dancing skeletons” - pulleys & wire giving motion to the bones of cognition.
- andai 2y ago
  But why stop there? All matter and all life is just increasingly fancy machines. https://en.wikipedia.org/wiki/Philosophical_zombie https://en.wikipedia.org/wiki/Philosophical_zombie Editor's note: I do not promote such a worldview -- my intention is precisely the opposite.
sgt101 2y ago
Our brains have effects that proceeded language. Look at lions for an example. We are much more (and a little less) than lions in terms of mind.
andai 2y ago
When I talk to 8B models, it's often painfully clear that they are operating mostly (entirely?) on the level of language. They often say things that make no sense except from a "word association" perspective. With bigger (400B models) that's not so clear. It would be silly to say that a fruit fly has the same thoughts as me, only a million times smaller quantitatively. I imagine the same thing is true (genuine qualitative leaps) in the 8B -> 400B direction.

Torkel 2y ago

Multimodal models are not limited by semantic understanding though, right? They are given photos, video, audio.

elzbardico 2y ago
For them, it is all a stream of tokens, roughly speaking.

mbil 2y ago

I might expect some extra-semantic cognitive faculties to emerge from LLMs, or at least be approximated by LLMs. Let me try to explain why. One example of extra-semantic ability is spatial reasoning. I can point to a spot on the ground and my dog will walk over to it — he’s probably not using semantic processing to talk through his relationship with the ground, the distance of each pace, his velocity, etc. But could a robotic dog powered by an LLM use a linguistic or symbolic representation of spacial concepts and actions to translate semantic reasoning into spacial reasoning? Imagine sensors with a measurement to language translation layer (“kitchen is five feet in front of you”), and actuators that can be triggered with language (“move forward two feet”). It seems conceivable that a detailed enough representation of the world, expressive enough controls, and a powerful enough LLM could result in something that is akin to spacial reasoning (an extra-semantic process), while under the hood it’s “just” semantic understanding.

defrost 2y ago
Spatial reasoning is more akin to visualising a 3D "odd shaped" fuel tank from 2D schematics and being able to mentally rotate that shape to estimate where a fluid line would be at various angles. This is distinct from stringing together treasure map instructions in an chain.
- throwuxiytayq 2y ago
  Isn’t spatial navigation a bit like graph walking, though? Also, AFAIK blind people describe it completely differently, and they’re generally confused by the whole concept of 3D perspective and objects getting visually smaller over distance, and so on. Brains don’t work the same for everyone in our species, and I wouldn’t presume to know the full internal representation just based on qualia.
- 082349872349872 2y ago
  I'm always impressed by the "straightedge-and-compass"-flavoured techniques drafters of old used to rotate views of odd 3D shapes from pairs of 2D schematics, in the centuries before CAD software. https://en.wikipedia.org/wiki/Descriptive_geometry#Finding_the_shortest_connector_between_two_given_skew_lines_PR_and_SU https://en.wikipedia.org/wiki/Descriptive_geometry#Finding_t... https://en.wikipedia.org/wiki/Gaspard_Monge#1789_and_after https://en.wikipedia.org/wiki/Gaspard_Monge#1789_and_after

ninetyninenine 2y ago

Do you have any evidence that human cognition (for speaking) is more than just an ability to string sentences together? Do you have any evidence that LLMs don't reason at all? A perfect machine designed to only string sentences together as perfect responses with no reasoning built it IS Indistinguishable from a machine that only builds sentences from pure reasoning. Either way nobody understands what's going on in the human brain and nobody understands why LLMs work. You don't know. You're just stating a belief.

slashdave 2y ago
There is evidence that many animals have reasoning abilities. Few can talk.
- ninetyninenine 2y ago
  There's also counter evidence that animals lack reasoning abilities. Ever see a gorilla attack it's own reflection or a dog chase it's own tail? Contradictory evidence displaying the ability to reason and the lack of the ability to reason is what is displayed by animals. Figures that the LLM displays the same contradictory evidence. But none of this evidence proves anything definitively. Much like human cognition, The LLM is a machine that we built but don't understand. No definitive statement can be made about it.
  - BriggyDwiggs42 2y ago
    You just argued that, because some animals display errors in reasoning, we can call into doubt the claim that any animal reasons. This does not follow. The reflection test, for example, is passed by some animals; we can test it by putting a red dot on their face and seeing if they mess with it. Either way, it’s not even a test of general reasoning abilities. I think you’re giving animals far too little credit.
    - sibeliuss 2y ago
      Not to mention all of the developments in mirror testing to account for differences in perception. What they're finding is that self-recognition is more common than assumed.

ekianjo 2y ago

> Human cognition is much more than just our ability to string sentences together animals without similar language capabilities dont seem to be too strong at reasoning. it could well be that language and reasoning are heavily linked together