3 ms·

> What the hell, we are looking at essentially millions of matrix multiplications done on powerful GPUs. And we're told this is what the brain does. Which is su

> What the hell, we are looking at essentially millions of matrix multiplications done on powerful GPUs. And we're told this is what the brain does. Which is supremely stupid, since as Bernardo Kastrup has emphasized many times, our brains are completely UNLIKE those systems. That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a transformer. These transformers are learning how to reproduce functions by inferring associations between inputs and outputs. The data whose function is being reproduced is the function in the human brain that produces human text. Therefore these transformers are literally learning how the human brain generates text, ie. this is what the brain does. Furthermore, there have been neuroscience papers correlating spiking behaviour in neural networks and human neurons when processing the same information, lending even more credence to this hypothesis. It's not as far-fetched as you think.

quonn 4y ago
> That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a transformer. That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain. Linux is written in C and that‘s Turing complete. And? Therefore Linux is like the brain?
- naasking 4y ago
  > That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain. It's not irrelevant, it was merely the premise for the following point, which was that transformers are trained to learn a function. Since they're Turing complete, they can thus hypothetically implement any function, including the function governing human speech, as I described. If transformers were not Turing complete, then they arguably would not be computationally powerful enough to model the brain which we know is Turing complete. What is irrelevant is trying to dismiss transformers as intelligent, or like the brain, simply because they're "merely matrix multiplications".
  - uoaei 4y ago
    What is your basis for asserting that transformers (not even sure what's meant: one layer?) are Turing-complete?
    - naasking 4y ago
      It's a well known property: https://arxiv.org/abs/1901.03429 https://arxiv.org/abs/1901.03429
  - lm28469 4y ago
    > What is irrelevant is trying to dismiss transformers as intelligent, or like the brain, Well there is a very very very big thing that makes it relevant, which would be the fact that we have no fucking clue on how our brain actually works. Wild claims, as always, no sources, as always
    - naasking 4y ago
      Thanks for agreeing with my point, which is that all of the people claiming that these systems are NOT like the human brain have no clue what they're talking about, and that these systems very well could operate just like the brain.
      - HopenHeyHi 4y ago
        Winner winner chicken dinner. Could sum the argument as: Transformers appear to be approximately emulating the brain despite us not fully understanding how either works "inside". Which is weird and eerie, but plausible.
      - lm28469 4y ago
        One smart ape once said: extraordinary claims require extraordinary evidence The burden of proof is on you, and so far you produced nothing of value in that regard
        naasking 4y ago
lm28469 4y ago
Do you have like any sources to back up you magical claims ? The more I look into the more I see a cult/religion more than a science
- naasking 4y ago
  What claims, specifically, do you want elaborated?
- MacsHeadroom 4y ago
  Transformers are turing complete: https://arxiv.org/abs/1901.03429 https://arxiv.org/abs/1901.03429 Additionally, Transformers are RNNs: https://arxiv.org/abs/2006.16236 https://arxiv.org/abs/2006.16236 Transformers are turing complete RNNs, as are the Neocortex. There are differences like spiking vs non-spiking neurons, universal vs non-universal activation, the effects of brain waves, bicameralism, etc. But these differences may or may not be important depending on the topic of discussion. Additionally, the differences increasingly disappear as transformers are increasingly augmented with functionally equivalent systems.

quonn 4y ago

> That's irrelevant. Transformers are Turing complete. Whatever mathematical system underlies human cognition will thus have some expression as a transformer. That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain. Linux is written in C and that‘s Turing complete. And? Therefore Linux is like the brain?

naasking 4y ago
> That‘s irrelevant, too, because nothing indicates that the actual transformer of, say, GPT or the training is in any way connected to such an expression of the human brain. It's not irrelevant, it was merely the premise for the following point, which was that transformers are trained to learn a function. Since they're Turing complete, they can thus hypothetically implement any function, including the function governing human speech, as I described. If transformers were not Turing complete, then they arguably would not be computationally powerful enough to model the brain which we know is Turing complete. What is irrelevant is trying to dismiss transformers as intelligent, or like the brain, simply because they're "merely matrix multiplications".
- uoaei 4y ago
  What is your basis for asserting that transformers (not even sure what's meant: one layer?) are Turing-complete?
  - naasking 4y ago
    It's a well known property: https://arxiv.org/abs/1901.03429 https://arxiv.org/abs/1901.03429
- lm28469 4y ago
  > What is irrelevant is trying to dismiss transformers as intelligent, or like the brain, Well there is a very very very big thing that makes it relevant, which would be the fact that we have no fucking clue on how our brain actually works. Wild claims, as always, no sources, as always
  - naasking 4y ago
    Thanks for agreeing with my point, which is that all of the people claiming that these systems are NOT like the human brain have no clue what they're talking about, and that these systems very well could operate just like the brain.
    - HopenHeyHi 4y ago
      Winner winner chicken dinner. Could sum the argument as: Transformers appear to be approximately emulating the brain despite us not fully understanding how either works "inside". Which is weird and eerie, but plausible.
    - lm28469 4y ago
      One smart ape once said: extraordinary claims require extraordinary evidence The burden of proof is on you, and so far you produced nothing of value in that regard
      - naasking 4y ago

lm28469 4y ago

Do you have like any sources to back up you magical claims ? The more I look into the more I see a cult/religion more than a science

naasking 4y ago
What claims, specifically, do you want elaborated?
MacsHeadroom 4y ago
Transformers are turing complete: https://arxiv.org/abs/1901.03429 https://arxiv.org/abs/1901.03429 Additionally, Transformers are RNNs: https://arxiv.org/abs/2006.16236 https://arxiv.org/abs/2006.16236 Transformers are turing complete RNNs, as are the Neocortex. There are differences like spiking vs non-spiking neurons, universal vs non-universal activation, the effects of brain waves, bicameralism, etc. But these differences may or may not be important depending on the topic of discussion. Additionally, the differences increasingly disappear as transformers are increasingly augmented with functionally equivalent systems.