4 ms·

> You would realize that all we are doing is using statistics to predict what the next item might be. So what does "understanding" really means then? "Underst

> You would realize that all we are doing is using statistics to predict what the next item might be. So what does "understanding" really means then? "Understanding" is not really well defined. Either we (humans) do it, but then LLMs might just do it as well, depending on the definition of "understanding", or we both don't do it. But if no-one is really "understanding" anything, then this definition of "understanding" is maybe not useful.

dboreham 3y ago
Bingo
Joeri 3y ago
Nobody actually understands how humans actually understand something, just like nobody actually understands how LLMs do what they do. Everybody opining about it is doing just that: offering an opinion. Geoff Hinton’s opinion is worth more than someone else’s, but it is still an opinion.
- im_down_w_otp 3y ago
  I don’t know about the human part, but we absolutely understand how LLMs do what they do. They’re not magic.
  - famouswaffles 3y ago
    No we don't. No it's not "magic". No we don't understand what the black box is doing.
    - brookst 3y ago
      For some values of “we”
      - famouswaffles 3y ago
        For every value of we. "I understand the internals of GPT" is the fastest way to demonstrate you have no idea what you're talking about.
  - chpatrick 3y ago
    We understand the architecture but we don't understand the weights.
  - usrbinbash 3y ago
    We also understand, down to a very very very microscopic level, how neurons work. We also know a helluva lot about the architecture of the brain. Does that mean we can explain our own intelligence, how our minds actually work? Nope.
deleted 3y ago
[deleted]
mannykannot 3y ago
I feel that LLMs raise some very interesting challenges for anyone trying to figure out what it means to understand something and how we do it, but I am not yet ready to agree with Hinton. For example, we are aware that some, but by no means all, of what people say is about an external world that may or may not conform to what the words say. We can also doubt that we have understood things correctly, and take steps to either confirm or refute our opinions. We see ourselves as entities in an external reality containing other individuals who also do this, and that we, and they, have a limited ability to influence what happens in that world. Do LLMs do these things, or is what they produce a result of having a lot of information about the purely formal properties of human language use, independently of semantics?
- hackinthebochs 3y ago
  >I feel that LLMs raise some very interesting challenges for anyone trying to figure out what it means to understand something and how we do it, but I am not yet ready to agree with Hinton. Agreed. What LLMs say about understanding deserves a lot more attention than it has received. I wrote down some of my thoughts on the matter: https://www.reddit.com/r/naturalism/comments/1236vzf https://www.reddit.com/r/naturalism/comments/1236vzf >Do LLMs do these things, or is what they produce a result of having a lot of information about the purely formal properties of human language use, independently of semantics? These two points aren't necessarily in opposition, and understanding why is I think key to solving a lot of important problems around intelligence, sentience, etc. To compute is to operate on formal properties. But this doesn't exclude semantic properties from having causal relevance to the behavior of the system. What we need is a way to conceptualize how a system can have multiple related descriptions at different levels. A description at the level of semantics doesn't exclude a description in terms of formal properties or vice versa. I think of it in terms of constraints: the higher level descriptions constrain the lower level behavior. What the computational description does is ensures the higher level semantic constraint is maintained. It does this by the particular space of computational dynamics it follows. Essentially, the information that picks out this programs space of branching dynamics embeds the semantic description in question, and this description realizes the computational dynamic necessary to maintain the higher level semantic constraint. Rather than semantics being in opposition to formal properties, they are two sides of the same coin.

dboreham 3y ago

Bingo

Joeri 3y ago

Nobody actually understands how humans actually understand something, just like nobody actually understands how LLMs do what they do. Everybody opining about it is doing just that: offering an opinion. Geoff Hinton’s opinion is worth more than someone else’s, but it is still an opinion.

im_down_w_otp 3y ago
I don’t know about the human part, but we absolutely understand how LLMs do what they do. They’re not magic.
- famouswaffles 3y ago
  No we don't. No it's not "magic". No we don't understand what the black box is doing.
  - brookst 3y ago
    For some values of “we”
    - famouswaffles 3y ago
      For every value of we. "I understand the internals of GPT" is the fastest way to demonstrate you have no idea what you're talking about.
- chpatrick 3y ago
  We understand the architecture but we don't understand the weights.
- usrbinbash 3y ago
  We also understand, down to a very very very microscopic level, how neurons work. We also know a helluva lot about the architecture of the brain. Does that mean we can explain our own intelligence, how our minds actually work? Nope.

deleted 3y ago

[deleted]

mannykannot 3y ago

I feel that LLMs raise some very interesting challenges for anyone trying to figure out what it means to understand something and how we do it, but I am not yet ready to agree with Hinton. For example, we are aware that some, but by no means all, of what people say is about an external world that may or may not conform to what the words say. We can also doubt that we have understood things correctly, and take steps to either confirm or refute our opinions. We see ourselves as entities in an external reality containing other individuals who also do this, and that we, and they, have a limited ability to influence what happens in that world. Do LLMs do these things, or is what they produce a result of having a lot of information about the purely formal properties of human language use, independently of semantics?

hackinthebochs 3y ago
>I feel that LLMs raise some very interesting challenges for anyone trying to figure out what it means to understand something and how we do it, but I am not yet ready to agree with Hinton. Agreed. What LLMs say about understanding deserves a lot more attention than it has received. I wrote down some of my thoughts on the matter: https://www.reddit.com/r/naturalism/comments/1236vzf https://www.reddit.com/r/naturalism/comments/1236vzf >Do LLMs do these things, or is what they produce a result of having a lot of information about the purely formal properties of human language use, independently of semantics? These two points aren't necessarily in opposition, and understanding why is I think key to solving a lot of important problems around intelligence, sentience, etc. To compute is to operate on formal properties. But this doesn't exclude semantic properties from having causal relevance to the behavior of the system. What we need is a way to conceptualize how a system can have multiple related descriptions at different levels. A description at the level of semantics doesn't exclude a description in terms of formal properties or vice versa. I think of it in terms of constraints: the higher level descriptions constrain the lower level behavior. What the computational description does is ensures the higher level semantic constraint is maintained. It does this by the particular space of computational dynamics it follows. Essentially, the information that picks out this programs space of branching dynamics embeds the semantic description in question, and this description realizes the computational dynamic necessary to maintain the higher level semantic constraint. Rather than semantics being in opposition to formal properties, they are two sides of the same coin.