5 ms·

The LLMs are predictive text. They only know that a hungry cat wants food because there are enough examples of that scenario on the Web. Related scenarios lead

The LLMs are predictive text. They only know that a hungry cat wants food because there are enough examples of that scenario on the Web. Related scenarios lead to nonsense like: the cat may want to go on a walk. I’m not making this up, ChatGPT recently suggested that I take my cat on a walk. This is likely bleeding too many concepts together (dog=pet=cat) and represents over fitting IMO. Cosine distance can only do so well, it’s not good at distinguishing nuance. But it’ll gladly regurgitate a rephrased Wikipedia article.

darkerside 4y ago
Seems like something an intelligent child who's never owned a cat might suggest
- mach1ne 4y ago
  The difference between LLM cognition and human cognition probably lies somewhere in this dichotomy.
kilgnad 4y ago
Wrong. There is science around the fact that chatGPT builds higher level macro structures in the neural network. These structures have been shown to be an actual model of things within the world. This isn't even a debate anymore between two guys on the internet. There is literal science behind this and the difference between us is one person is behind on the science. But forget the science. There is even chatGPT output that literally shows it understood what it was told. It is clear chatGPT does not have perfect understanding of the world. It does create really stupid output. But this is ignoring the fact that there are tons of answers it gives that show unmistakably that it knows what you asked it.
- deleted 4y ago
  [deleted]
- Jensson 4y ago
  You don't think that complex structures are required to generate well formed text? Human language is extremely context dependent, and dealing with that is what all those macro structures does. Context dependence is literally the novel thing with transformers, they are context dependent statistical models to generate next words, make it huge and feed it a page of context and it can map those to a page of output to match that context.
  - naasking 4y ago
    Context is arguably understanding.
- afpx 4y ago
  Where can I learn more about these macro structures?
  - kilgnad 4y ago
    - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559 There's more. People are just in denial.
    - Godel_unicode 4y ago
      Your second bullet isn’t what the paper says. It says that the model’s accuracy under in-context learning from the meta gradients is similar to manual fine-tuning with back propagation (under some surprising constraints in the name of “fairness”). The language model isn’t implementing anything, you have the directionality wrong. GPT models are constructed with pretrained gradients which are applicable in a large set of situations. It’s just an optimization technique, albeit a clever one. Quoting from the paper: In summary, we explain ICL as a process of meta-optimization: (1) a Transformer-based pre- trained language model serves as a meta-optimizer; (2) it produces meta-gradients according to the demonstration examples through forward computa- tion; (3) through attention, the meta-gradients are applied to the original language model to build an ICL model.
      - kilgnad 4y ago
        What you quoted from the paper literally said what I said. There's no difference. The forward computation is computing gradients. These new gradients are applied to the model, to build the icl model via "attention". What I said: "implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples):" Perhaps you're not getting it. Forward computation is the neural network processing an input. The paper is saying gradients are built here. These new gradients are then applied through the transformers "attention" step. The icl model is the model with the attention applies. It is literally just another perspective of what transformers do.
- deleted 4y ago
  [deleted]
funcDropShadow 4y ago
> the cat may want to go on a walk Just for reference, I've been on a walk with my cat just yesterday ;-). Bengal cats do like going for a walk. My cat follows me without a leash. And he even begs to go for a walk once in a while.

darkerside 4y ago

Seems like something an intelligent child who's never owned a cat might suggest

mach1ne 4y ago
The difference between LLM cognition and human cognition probably lies somewhere in this dichotomy.

kilgnad 4y ago

Wrong. There is science around the fact that chatGPT builds higher level macro structures in the neural network. These structures have been shown to be an actual model of things within the world. This isn't even a debate anymore between two guys on the internet. There is literal science behind this and the difference between us is one person is behind on the science. But forget the science. There is even chatGPT output that literally shows it understood what it was told. It is clear chatGPT does not have perfect understanding of the world. It does create really stupid output. But this is ignoring the fact that there are tons of answers it gives that show unmistakably that it knows what you asked it.

deleted 4y ago
[deleted]
Jensson 4y ago
You don't think that complex structures are required to generate well formed text? Human language is extremely context dependent, and dealing with that is what all those macro structures does. Context dependence is literally the novel thing with transformers, they are context dependent statistical models to generate next words, make it huge and feed it a page of context and it can map those to a page of output to match that context.
- naasking 4y ago
  Context is arguably understanding.
afpx 4y ago
Where can I learn more about these macro structures?
- kilgnad 4y ago
  - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples): https://arxiv.org/abs/2212.10559 https://arxiv.org/abs/2212.10559 There's more. People are just in denial.
  - Godel_unicode 4y ago
    Your second bullet isn’t what the paper says. It says that the model’s accuracy under in-context learning from the meta gradients is similar to manual fine-tuning with back propagation (under some surprising constraints in the name of “fairness”). The language model isn’t implementing anything, you have the directionality wrong. GPT models are constructed with pretrained gradients which are applicable in a large set of situations. It’s just an optimization technique, albeit a clever one. Quoting from the paper: In summary, we explain ICL as a process of meta-optimization: (1) a Transformer-based pre- trained language model serves as a meta-optimizer; (2) it produces meta-gradients according to the demonstration examples through forward computa- tion; (3) through attention, the meta-gradients are applied to the original language model to build an ICL model.
    - kilgnad 4y ago
      What you quoted from the paper literally said what I said. There's no difference. The forward computation is computing gradients. These new gradients are applied to the model, to build the icl model via "attention". What I said: "implementing a mini "neural network training algorithm" (gradient descent fine-tuning for given examples):" Perhaps you're not getting it. Forward computation is the neural network processing an input. The paper is saying gradients are built here. These new gradients are then applied through the transformers "attention" step. The icl model is the model with the attention applies. It is literally just another perspective of what transformers do.
deleted 4y ago
[deleted]

funcDropShadow 4y ago

> the cat may want to go on a walk Just for reference, I've been on a walk with my cat just yesterday ;-). Bengal cats do like going for a walk. My cat follows me without a leash. And he even begs to go for a walk once in a while.