3 ms·

No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process. No amount of

No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process. No amount of training will cause a transformer to magically sprout feedback paths or internal memory, or an ability to alter it's own weights, etc. Architecture matters. The best you can hope for an LLM is that training will converge on the best LLM generating process it can be, which can be great for in-distribution prediction, but lousy for novel reasoning tasks beyond the capability of the architecture.

famouswaffles 2y ago
>No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process. Go back a few evolutionary steps and sure you can. Most ANN architectures basically have relatively little to no biases baked in and the Transformer might be the most blank slate we've built yet. >No amount of training will cause a transformer to magically sprout feedback paths or internal memory, or an ability to alter it's own weights, etc. A transformer can perform any computation it likes in a forward pass and you can arbitrarily increase inference compute time with the token length. Feedback paths? Sure. Compute inefficient? Perhaps. Some extra programming around the Model to facilitate this ? Maybe but the architecture certainly isn't stopping you. Even if it couldn't, limited =/ trivial. The Human Brain is not Turing complete. Internal Memory ? Did you miss the memo ? Recurrency is overrated. Attention is all you need. That said, there are already state keeping language model architectures around. Altering weights ? Can a transformer continuously train ? Sure. It's not really compute efficient but architecture certainly doesn't prohibit it. >Architecture matters Compute Efficiency? Sure. What it is capable of learning? Not so much
- HarHarVeryFunny 2y ago
  > A transformer can perform any computation it likes in a forward pass No it can't. A transformer has a fixed number of layers - call it N. It performs N sequential steps of computation to derive it's output. If a computation requires > N steps, then a transformer most certainly can not perform it in a forward pass. FYI, "attention is all you need" has the implicit context of "if all you want to build is a language model". Attention is not all you need if what you actually want to build is a cognitive architecture.
  - famouswaffles 2y ago
    Transformer produce the next token by manipulating K hidden vectors per layer, one vector per preceding token. So yes you can increase compute length arbitrarily by increasing tokens. Those tokens don't have to carry any information to work. https://arxiv.org/abs/2310.02226 https://arxiv.org/abs/2310.02226 And again, human brains are clearly limited in the number of steps it can compute without writing something down. Limited =/ Trivial >FYI, "attention is all you need" has the implicit context of "if all you want to build is a language model". Great. Do you know what a "language model" is capable of in the limit ? No These top research labs aren't only working on Transformers as they currently exist but it doesn't make much sense to abandon a golden goose before it has hit a wall.
    - HarHarVeryFunny 2y ago
      You are confusing number of sequential steps with total amount of compute spent. The input sequence is processed in parallel, regardless of length, so number of tokens has no impact on number of sequential compute steps which is always N=layers. > Do you know what a "language model" is capable of in the limit ? Well, yeah, if the language model is an N-layer transformer ...
      - famouswaffles 2y ago
        Fair Enough. Then increase N (N is almost always increased when a model is scaled up) and train or write things down and continue. A limitless iteration machine (without external aid) is currently an idea of fiction. Brains can't do it so I'm not particularly worried if machines can't either.
        HarHarVeryFunny 2y ago
        Increasing number of layers isn't a smart way to solve it. It order to be able to reason effectively and efficiently the model needs to use as much, or as little, compute as needed for a given task. Completing "1+1=" should take less compute steps than "A winning sequence for white here is ...". This lack of "variable compute" is a widely recognized shortcoming of transformer-based LLMs, and there are plenty of others. The point apropos this thread is that you can't just train an LLM to be something that it is not. If the generating process required variable compute (maybe 1000's of steps) - e.g. to come up with a chess move - then no amount of training can make the LLM converge to model this generative process... the best it can do is to model the outcome of the generative process, not the process itself. The difference is that without having learnt the generative process, the model will fail when presented with a novel input that it didn't see during training, and therefore didn't memorize the "cheat sheet" answer for.
machiaweliczny 2y ago
How about spiders intelligence? They don’t even have brain

famouswaffles 2y ago

>No amount of training would cause a fly brain to be able to do what an octopus or bird brain can, or to model their behavioral generating process. Go back a few evolutionary steps and sure you can. Most ANN architectures basically have relatively little to no biases baked in and the Transformer might be the most blank slate we've built yet. >No amount of training will cause a transformer to magically sprout feedback paths or internal memory, or an ability to alter it's own weights, etc. A transformer can perform any computation it likes in a forward pass and you can arbitrarily increase inference compute time with the token length. Feedback paths? Sure. Compute inefficient? Perhaps. Some extra programming around the Model to facilitate this ? Maybe but the architecture certainly isn't stopping you. Even if it couldn't, limited =/ trivial. The Human Brain is not Turing complete. Internal Memory ? Did you miss the memo ? Recurrency is overrated. Attention is all you need. That said, there are already state keeping language model architectures around. Altering weights ? Can a transformer continuously train ? Sure. It's not really compute efficient but architecture certainly doesn't prohibit it. >Architecture matters Compute Efficiency? Sure. What it is capable of learning? Not so much

HarHarVeryFunny 2y ago
> A transformer can perform any computation it likes in a forward pass No it can't. A transformer has a fixed number of layers - call it N. It performs N sequential steps of computation to derive it's output. If a computation requires > N steps, then a transformer most certainly can not perform it in a forward pass. FYI, "attention is all you need" has the implicit context of "if all you want to build is a language model". Attention is not all you need if what you actually want to build is a cognitive architecture.
- famouswaffles 2y ago
  Transformer produce the next token by manipulating K hidden vectors per layer, one vector per preceding token. So yes you can increase compute length arbitrarily by increasing tokens. Those tokens don't have to carry any information to work. https://arxiv.org/abs/2310.02226 https://arxiv.org/abs/2310.02226 And again, human brains are clearly limited in the number of steps it can compute without writing something down. Limited =/ Trivial >FYI, "attention is all you need" has the implicit context of "if all you want to build is a language model". Great. Do you know what a "language model" is capable of in the limit ? No These top research labs aren't only working on Transformers as they currently exist but it doesn't make much sense to abandon a golden goose before it has hit a wall.
  - HarHarVeryFunny 2y ago
    You are confusing number of sequential steps with total amount of compute spent. The input sequence is processed in parallel, regardless of length, so number of tokens has no impact on number of sequential compute steps which is always N=layers. > Do you know what a "language model" is capable of in the limit ? Well, yeah, if the language model is an N-layer transformer ...
    - famouswaffles 2y ago
      Fair Enough. Then increase N (N is almost always increased when a model is scaled up) and train or write things down and continue. A limitless iteration machine (without external aid) is currently an idea of fiction. Brains can't do it so I'm not particularly worried if machines can't either.
      - HarHarVeryFunny 2y ago
        Increasing number of layers isn't a smart way to solve it. It order to be able to reason effectively and efficiently the model needs to use as much, or as little, compute as needed for a given task. Completing "1+1=" should take less compute steps than "A winning sequence for white here is ...". This lack of "variable compute" is a widely recognized shortcoming of transformer-based LLMs, and there are plenty of others. The point apropos this thread is that you can't just train an LLM to be something that it is not. If the generating process required variable compute (maybe 1000's of steps) - e.g. to come up with a chess move - then no amount of training can make the LLM converge to model this generative process... the best it can do is to model the outcome of the generative process, not the process itself. The difference is that without having learnt the generative process, the model will fail when presented with a novel input that it didn't see during training, and therefore didn't memorize the "cheat sheet" answer for.

machiaweliczny 2y ago

How about spiders intelligence? They don’t even have brain