3 ms·

> Human minds have semantic relationships to the rest of the world that LLMs do not have. This is a bold claim that seems built on a presumption of mind-body d

> Human minds have semantic relationships to the rest of the world that LLMs do not have. This is a bold claim that seems built on a presumption of mind-body dualism. Brains don't have semantic relationships with anything. They are neurons hooked up to sensors and actuators. Any inferences they produce are the result of statistical processes.

pdonis 4y ago
> This is a bold claim that seems built on a presumption of mind-body dualism. Not at all. I'm a physicalist; I don't believe the mind is a separate thing from the brain. > Brains don't have semantic relationships with anything. Yes, they do: you describe them yourself: > They are neurons hooked up to sensors and actuators. Those are semantic relationships with the rest of the world. Although your short description does not by any means do justice to the complexity and richness of those relationships.
- strbean 4y ago
  I see what you're saying; ChatGPT doesn't have a physical relationship with the world, doesn't have agency (is essentially paused until given input), doesn't have reward/punishment stimulae, etc. I do think that a large portion of what seems to be missing here is trivial to add, relative to the effort in creating ChatGPT in the first place. Side note: I'm not sure 'semantic relationship' is the right term here. Pretty sure it is specific to relationships between linguistic constructs. That wording very much triggered my "Bah, dualism!" response, as I thought you were insinuating some metaphysical bond between the mind and the world. Maybe "meaningful relationship" would serve better?
  - pdonis 4y ago
    > I do think that a large portion of what seems to be missing here is trivial to add If you really think it's trivial, then do it! I would be interested to see the results of anyone doing this. But there aren't any to see right now. > I'm not sure 'semantic relationship' is the right term here. It might not be; but in the cognitive science literature that term is used for more than just relationships between linguistic constructs; it is used for relationships between internal features of a model or an entity and features of the external world. I think this usage is also common in robotics, and more generally in domains like mechanical engineering which are often concerned with creating software programs to do things like manage fuel and air flow in car engines.
    - fidgewidge 4y ago
      Look again. There are papers that hook up llms to robots with vision and other sensors. The LLM is fed descriptions of the world and then emits instructions for where to go
      - strbean 4y ago
        The thought of an LLM interacting with the world through a MUD is entertaining :)
- hackinthebochs 4y ago
  >Those are semantic relationships with the rest of the world. If this is all that counts as semantic relationships, then I see no reason why a language model doesn't have this kind of semantic relationship, albeit in a very different modality. Tokens and their co-occurrences are a kind of sensor to the world. In the same way we discover quantum mechanics by way of induction over indirect relationships among the signals incident to our perceptual apparatus (the sensors and actuators that translate external signals into internal signals), a language model could learn much about the world by way of induction over token co-occurrences. Sure, there are limits, conscious perception of the world being the big one, but I see no reason to think conscious perception of X is required to know or understand X.
  - pdonis 4y ago
    > I see no reason why a language model doesn't have this kind of semantic relationship One certainly could hook up a language model to sensors and actuators to give it semantic relationships with the rest of the world. But nobody has done this. And giving it semantic relationships of the same order of complexity and richness that human brains have is an extremely tall order, one I don't expect anyone to come anywhere close to doing any time soon. > Tokens and their co-occurrences are a kind of sensor to the world They can be a kind of extremely low bandwidth, low resolution sensor, yes. But for that to ground any kind of semantic relationship to the world, the model would need the ability to frame hypotheses about what this sensor data means, and test them by interacting with the world and seeing what the results were. No language model does that now.
    - hackinthebochs 4y ago
      So much rides on your implicit notion of semantic relationship, but this dependence needs demonstration. The fact that some pattern of signals on my perceptual apparatus is caused by an apple in the real world does not mean that I have knowledge or understanding of an apple in virtue of this causal relation. That my sensory signals are caused by apples is an accident of this world, one we are completely blind to. If all apples in the world were swapped with fapples (fake apples), where all sensory experiences that have up to now been caused by apples are now caused by fapples, we would be none the wiser. The semantics (i.e. wide content) of our perceptual experiences is irrelevant to literally everything we know and how we interact with the world. Our knowledge of the world is limited to our sensory experiences and our deductions, inferences, etc derived from our sensory experiences. Our situatedness in the world is only relevant insofar as it entails the space of our sensory experiences. >the model would need the ability to frame hypotheses about what this sensor data means, and test them by interacting with the world and seeing what the results were. Why do we need to actively test our model to come to an understanding of the world? Yes, that is how we biological organisms happen to learn about the world. But it is not clear that it is required. Language models learn by developing internal models that predict the next token. But this prediction implicitly generates representations of the processes that generate the tokens. There is no in principle limit to the resolution of this model given a sufficiently large and diverse training set.
pdonis 4y ago
> Any inferences they produce are the result of statistical processes. I don't know how you would justify this claim since we don't know nearly enough about how the brain actually does things like make inferences.
- strbean 4y ago
  But we know enough about fundamental physics to know that all physical processes are statistical ones.
  - pdonis 4y ago
    We know no such thing. Some interpretations of QM make this claim, but there is no consensus on whether those interpretations are correct.
- uh_uh 4y ago
  What else would it be? Are you suggesting some non-physical process?

pdonis 4y ago

> This is a bold claim that seems built on a presumption of mind-body dualism. Not at all. I'm a physicalist; I don't believe the mind is a separate thing from the brain. > Brains don't have semantic relationships with anything. Yes, they do: you describe them yourself: > They are neurons hooked up to sensors and actuators. Those are semantic relationships with the rest of the world. Although your short description does not by any means do justice to the complexity and richness of those relationships.

strbean 4y ago
I see what you're saying; ChatGPT doesn't have a physical relationship with the world, doesn't have agency (is essentially paused until given input), doesn't have reward/punishment stimulae, etc. I do think that a large portion of what seems to be missing here is trivial to add, relative to the effort in creating ChatGPT in the first place. Side note: I'm not sure 'semantic relationship' is the right term here. Pretty sure it is specific to relationships between linguistic constructs. That wording very much triggered my "Bah, dualism!" response, as I thought you were insinuating some metaphysical bond between the mind and the world. Maybe "meaningful relationship" would serve better?
- pdonis 4y ago
  > I do think that a large portion of what seems to be missing here is trivial to add If you really think it's trivial, then do it! I would be interested to see the results of anyone doing this. But there aren't any to see right now. > I'm not sure 'semantic relationship' is the right term here. It might not be; but in the cognitive science literature that term is used for more than just relationships between linguistic constructs; it is used for relationships between internal features of a model or an entity and features of the external world. I think this usage is also common in robotics, and more generally in domains like mechanical engineering which are often concerned with creating software programs to do things like manage fuel and air flow in car engines.
  - fidgewidge 4y ago
    Look again. There are papers that hook up llms to robots with vision and other sensors. The LLM is fed descriptions of the world and then emits instructions for where to go
    - strbean 4y ago
      The thought of an LLM interacting with the world through a MUD is entertaining :)
hackinthebochs 4y ago
>Those are semantic relationships with the rest of the world. If this is all that counts as semantic relationships, then I see no reason why a language model doesn't have this kind of semantic relationship, albeit in a very different modality. Tokens and their co-occurrences are a kind of sensor to the world. In the same way we discover quantum mechanics by way of induction over indirect relationships among the signals incident to our perceptual apparatus (the sensors and actuators that translate external signals into internal signals), a language model could learn much about the world by way of induction over token co-occurrences. Sure, there are limits, conscious perception of the world being the big one, but I see no reason to think conscious perception of X is required to know or understand X.
- pdonis 4y ago
  > I see no reason why a language model doesn't have this kind of semantic relationship One certainly could hook up a language model to sensors and actuators to give it semantic relationships with the rest of the world. But nobody has done this. And giving it semantic relationships of the same order of complexity and richness that human brains have is an extremely tall order, one I don't expect anyone to come anywhere close to doing any time soon. > Tokens and their co-occurrences are a kind of sensor to the world They can be a kind of extremely low bandwidth, low resolution sensor, yes. But for that to ground any kind of semantic relationship to the world, the model would need the ability to frame hypotheses about what this sensor data means, and test them by interacting with the world and seeing what the results were. No language model does that now.
  - hackinthebochs 4y ago
    So much rides on your implicit notion of semantic relationship, but this dependence needs demonstration. The fact that some pattern of signals on my perceptual apparatus is caused by an apple in the real world does not mean that I have knowledge or understanding of an apple in virtue of this causal relation. That my sensory signals are caused by apples is an accident of this world, one we are completely blind to. If all apples in the world were swapped with fapples (fake apples), where all sensory experiences that have up to now been caused by apples are now caused by fapples, we would be none the wiser. The semantics (i.e. wide content) of our perceptual experiences is irrelevant to literally everything we know and how we interact with the world. Our knowledge of the world is limited to our sensory experiences and our deductions, inferences, etc derived from our sensory experiences. Our situatedness in the world is only relevant insofar as it entails the space of our sensory experiences. >the model would need the ability to frame hypotheses about what this sensor data means, and test them by interacting with the world and seeing what the results were. Why do we need to actively test our model to come to an understanding of the world? Yes, that is how we biological organisms happen to learn about the world. But it is not clear that it is required. Language models learn by developing internal models that predict the next token. But this prediction implicitly generates representations of the processes that generate the tokens. There is no in principle limit to the resolution of this model given a sufficiently large and diverse training set.

pdonis 4y ago

> Any inferences they produce are the result of statistical processes. I don't know how you would justify this claim since we don't know nearly enough about how the brain actually does things like make inferences.

strbean 4y ago
But we know enough about fundamental physics to know that all physical processes are statistical ones.
- pdonis 4y ago
  We know no such thing. Some interpretations of QM make this claim, but there is no consensus on whether those interpretations are correct.
uh_uh 4y ago
What else would it be? Are you suggesting some non-physical process?