6 ms·

I think LLMs understand things in abstract. e.g. they know that if a 'Cat' is 'Miaowing' then if you 'give' it some 'cat food' it will then be 'happy'. It doesn

I think LLMs understand things in abstract. e.g. they know that if a 'Cat' is 'Miaowing' then if you 'give' it some 'cat food' it will then be 'happy'. It doesn't know what any of those things _are_, but it knows how X and Y and Z relate. And so in the abstract sense it knows that 'hunger' causes a 'cat' to 'miaow' it will still have no awareness We dont know how awareness works, so we're not in a position to say what has and hasn't got it. Think about the milliseconds in which ChatGPT uses its 4000 token length to analyse approx 2000 words all at same time, in a process encompassing a massive number of GPUs processing a mind boggling number of parameters. How are we to say what it going on for those milliseconds? There could be some sort of abstract analogue to awareness happening there for a burst of a few milliseconds. These LLMs are all about emergent effects from huge scale. Similarly, me and you are made out of 30 trillion microscopic dumb biological robots, none of whom know who we are or care, but nonetheless, we have awareness.

breckenedge 4y ago
The LLMs are predictive text. They only know that a hungry cat wants food because there are enough examples of that scenario on the Web. Related scenarios lead to nonsense like: the cat may want to go on a walk. I’m not making this up, ChatGPT recently suggested that I take my cat on a walk. This is likely bleeding too many concepts together (dog=pet=cat) and represents over fitting IMO. Cosine distance can only do so well, it’s not good at distinguishing nuance. But it’ll gladly regurgitate a rephrased Wikipedia article.
- darkerside 4y ago
  Seems like something an intelligent child who's never owned a cat might suggest
  - mach1ne 4y ago
    The difference between LLM cognition and human cognition probably lies somewhere in this dichotomy.
- kilgnad 4y ago
  Wrong. There is science around the fact that chatGPT builds higher level macro structures in the neural network. These structures have been shown to be an actual model of things within the world. This isn't even a debate anymore between two guys on the internet. There is literal science behind this and the difference between us is one person is behind on the science. But forget the science. There is even chatGPT output that literally shows it understood what it was told. It is clear chatGPT does not have perfect understanding of the world. It does create really stupid output. But this is ignoring the fact that there are tons of answers it gives that show unmistakably that it knows what you asked it.
  - deleted 4y ago
    [deleted]
  - Jensson 4y ago
    You don't think that complex structures are required to generate well formed text? Human language is extremely context dependent, and dealing with that is what all those macro structures does. Context dependence is literally the novel thing with transformers, they are context dependent statistical models to generate next words, make it huge and feed it a page of context and it can map those to a page of output to match that context.
    - naasking 4y ago
      Context is arguably understanding.
  - afpx 4y ago
    Where can I learn more about these macro structures?
    - kilgnad 4y ago
      - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559 There's more. People are just in denial.
- funcDropShadow 4y ago
  > the cat may want to go on a walk Just for reference, I've been on a walk with my cat just yesterday ;-). Bengal cats do like going for a walk. My cat follows me without a leash. And he even begs to go for a walk once in a while.
Retric 4y ago
You’re anthropomorphizing these chat bots. They may associate those words in specific contexts, but they don’t have that kind of abstract understanding. The best way to understand them isn’t to look at what they do well but where they fail. A great example is how ChatGPT can initially make a few chess moves that seem reasonable, but very quickly it stops making valid moves. It’s not operating from some model of the game but rather imitating sequences of moves it’s seen before. The best analogy isn’t cognition but someone trying to make a better version of “lorem ipsum” for whatever promo you’re giving it.
- 2OEH8eoCRo0 4y ago
  > You’re anthropomorphizing these chat bots. Oh? don't anthropomorphize the thing we are supposed to "chat" with? It's basically what they're designed for except when they rudely tell you "I'm just an LLM!"
- codeulike 4y ago
  GPTs can model simpler games like Othello https://thegradient.pub/othello/ https://thegradient.pub/othello/ https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 with enough training data it could probably model chess - not well enough to win but well enough to make legal moves You think that I am misunderstanding whats going on and anthropomorphising ChatGPT. I know how ChatGPT works, my position is that we might be overestimating our selves, and underestimating the power of emergent phenomenon.
  - kilgnad 4y ago
    Exactly. The science is starting to realize the emergent effects of LLMs. These things can literally learn simply by "looking over your shoulder". We thought that you had to explicitly program hierarchical structures of causal reasoning into the network but science is showing that these structures are emergent. People won't believe you if you show actual evidence. You have to throw them a scientific paper written by an "expert" lol. And even then they will find a hard time changing their viewpoint. Its so strange why people are trying to downplay it all when even the science is showing they're wrong. They have to throw accusations around of anthropomorphisation. Seriously? It's very easy to identify the bias of anthropomorphisation. Anyone can easily tiptoe around that bias with a simple argument. Clearly what's going on with chatGPT is much more complex then that. I recommend people stop using that word in this topic. It's akin to accusing someone they have brain damage. Clearly they don't.

breckenedge 4y ago

The LLMs are predictive text. They only know that a hungry cat wants food because there are enough examples of that scenario on the Web. Related scenarios lead to nonsense like: the cat may want to go on a walk. I’m not making this up, ChatGPT recently suggested that I take my cat on a walk. This is likely bleeding too many concepts together (dog=pet=cat) and represents over fitting IMO. Cosine distance can only do so well, it’s not good at distinguishing nuance. But it’ll gladly regurgitate a rephrased Wikipedia article.

darkerside 4y ago
Seems like something an intelligent child who's never owned a cat might suggest
- mach1ne 4y ago
  The difference between LLM cognition and human cognition probably lies somewhere in this dichotomy.
kilgnad 4y ago
Wrong. There is science around the fact that chatGPT builds higher level macro structures in the neural network. These structures have been shown to be an actual model of things within the world. This isn't even a debate anymore between two guys on the internet. There is literal science behind this and the difference between us is one person is behind on the science. But forget the science. There is even chatGPT output that literally shows it understood what it was told. It is clear chatGPT does not have perfect understanding of the world. It does create really stupid output. But this is ignoring the fact that there are tons of answers it gives that show unmistakably that it knows what you asked it.
- deleted 4y ago
  [deleted]
- Jensson 4y ago
  You don't think that complex structures are required to generate well formed text? Human language is extremely context dependent, and dealing with that is what all those macro structures does. Context dependence is literally the novel thing with transformers, they are context dependent statistical models to generate next words, make it huge and feed it a page of context and it can map those to a page of output to match that context.
  - naasking 4y ago
    Context is arguably understanding.
- afpx 4y ago
  Where can I learn more about these macro structures?
  - kilgnad 4y ago
    - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559 There's more. People are just in denial.
funcDropShadow 4y ago
> the cat may want to go on a walk Just for reference, I've been on a walk with my cat just yesterday ;-). Bengal cats do like going for a walk. My cat follows me without a leash. And he even begs to go for a walk once in a while.

Retric 4y ago

You’re anthropomorphizing these chat bots. They may associate those words in specific contexts, but they don’t have that kind of abstract understanding. The best way to understand them isn’t to look at what they do well but where they fail. A great example is how ChatGPT can initially make a few chess moves that seem reasonable, but very quickly it stops making valid moves. It’s not operating from some model of the game but rather imitating sequences of moves it’s seen before. The best analogy isn’t cognition but someone trying to make a better version of “lorem ipsum” for whatever promo you’re giving it.

2OEH8eoCRo0 4y ago
> You’re anthropomorphizing these chat bots. Oh? don't anthropomorphize the thing we are supposed to "chat" with? It's basically what they're designed for except when they rudely tell you "I'm just an LLM!"
codeulike 4y ago
GPTs can model simpler games like Othello https://thegradient.pub/othello/ https://thegradient.pub/othello/ https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 with enough training data it could probably model chess - not well enough to win but well enough to make legal moves You think that I am misunderstanding whats going on and anthropomorphising ChatGPT. I know how ChatGPT works, my position is that we might be overestimating our selves, and underestimating the power of emergent phenomenon.
- kilgnad 4y ago
  Exactly. The science is starting to realize the emergent effects of LLMs. These things can literally learn simply by "looking over your shoulder". We thought that you had to explicitly program hierarchical structures of causal reasoning into the network but science is showing that these structures are emergent. People won't believe you if you show actual evidence. You have to throw them a scientific paper written by an "expert" lol. And even then they will find a hard time changing their viewpoint. Its so strange why people are trying to downplay it all when even the science is showing they're wrong. They have to throw accusations around of anthropomorphisation. Seriously? It's very easy to identify the bias of anthropomorphisation. Anyone can easily tiptoe around that bias with a simple argument. Clearly what's going on with chatGPT is much more complex then that. I recommend people stop using that word in this topic. It's akin to accusing someone they have brain damage. Clearly they don't.