3 ms·

It's not even close to a mammal. It might have 1bn artificial neurons, while a cat has 700m biological neurons. But a) individual mammal neurons are far more c

It's not even close to a mammal. It might have 1bn artificial neurons, while a cat has 700m biological neurons. But a) individual mammal neurons are far more computationally complex than any existing artificial neuron, even in SNNs. b) a cat has 10 trillion synapses compared to the 100 billion artificial synapses in Intel's computer. So even ignoring how simple artificial neurons are, mammal brains have far more topological complexity.

hollerith 2y ago
But semiconductors operate so much faster than neurons that it is hard to be confident that AI hadware has not already caught up to cat brains.
- nicklecompte 2y ago
  Well, don't underestimate the sheer back-of-the-envelope power of a cat's brain: 700m neurons x 200Hz per neuron is a LOT of compute, like ~10,000 modern CPU cores in terms of ops/second. Pretty impressive when you consider the thermal efficiency.... Regardless, looking at AI power that way is a mistake. There are already a few supercomputers that can perform more operations/second than the human brain and I wouldn't be too surprised if this neuromorphic computer can perform more ops/s than a cat. But the structure of the brain is important: brains aren't just simply glued-together layers of neurons. The incredible topological complexity and integration of mammal brains surely means that certain "parallelizable algorithms" can be run much more efficiently, and that disparate information can be pulled together more readily for identifying patterns and making mental models. All mammals (and birds) are faster learners than any artificial neural network, and they understand ideas like object permanence and causality, which no AI is even close to getting right. It is reasonable to suppose that the structural complexity of mammal/bird brains is an important part why their cognitive abilities are so much greater than 2024 AI. And I really can't overemphasize how complex mammal neurons are compared to artificial ones: https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... It might not matter that individual biological neurons are slow when by themselves they're as sophisticated as deep artificial neural networks. A neuron might operate at 200Hz but it's not solving "what's the sum of these floating-point vectors?" like a CPU/GPU. It's solving something way more sophisticated: there was a paper recently arguing that humans had individual neurons for visually classifying 2 things, 3 things, etc. (Cats can do this "subitizing" almost as well as we can and probably have their own neurons for it, but their understanding of numbers seems shallower than a primate's.) So: - the artificial neurons are far simpler than cat neurons - the links between the artificial neurons in neuromorphic computers are far simpler than the links between cat neurons in cat brains - even in naive ops/second cats are not slouches Simply matching the number of cat neurons and waving hands about "fast semiconductors" is not enough for me to think this machine is as powerful as a cat's brain. (And even if Intel made a computer theoretically capable of emulating a cat, I suspect it'll take several decades to develop AI software which is sophisticated enough to do the job.)
  - abeppu 2y ago
    I agree with much of this, but I think we also need to acknowledge our own ignorance over - how much of the complexity of mammalian neurons is "incidental" complexity, due to also needing to be living, functioning cells, etc as versus "necessary" complexity needed to perform as well as they do - how much specialization between regions and the details of the larger "connectome" are, vs how much can be accomplished by enough repetition of the right local structure (e.g. Hawkins' "thousand brains" etc) > It is reasonable to suppose that the structural complexity of mammal/bird brains is an important part why their cognitive abilities are so much greater than 2024 AI. Statements like this seem a premature to me -- we haven't really had systems with a simple (e.g. repeating) large scale structure and efficient adapting durable-memory-and-compute to really evaluate with, right?

hollerith 2y ago

But semiconductors operate so much faster than neurons that it is hard to be confident that AI hadware has not already caught up to cat brains.

nicklecompte 2y ago
Well, don't underestimate the sheer back-of-the-envelope power of a cat's brain: 700m neurons x 200Hz per neuron is a LOT of compute, like ~10,000 modern CPU cores in terms of ops/second. Pretty impressive when you consider the thermal efficiency.... Regardless, looking at AI power that way is a mistake. There are already a few supercomputers that can perform more operations/second than the human brain and I wouldn't be too surprised if this neuromorphic computer can perform more ops/s than a cat. But the structure of the brain is important: brains aren't just simply glued-together layers of neurons. The incredible topological complexity and integration of mammal brains surely means that certain "parallelizable algorithms" can be run much more efficiently, and that disparate information can be pulled together more readily for identifying patterns and making mental models. All mammals (and birds) are faster learners than any artificial neural network, and they understand ideas like object permanence and causality, which no AI is even close to getting right. It is reasonable to suppose that the structural complexity of mammal/bird brains is an important part why their cognitive abilities are so much greater than 2024 AI. And I really can't overemphasize how complex mammal neurons are compared to artificial ones: https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... It might not matter that individual biological neurons are slow when by themselves they're as sophisticated as deep artificial neural networks. A neuron might operate at 200Hz but it's not solving "what's the sum of these floating-point vectors?" like a CPU/GPU. It's solving something way more sophisticated: there was a paper recently arguing that humans had individual neurons for visually classifying 2 things, 3 things, etc. (Cats can do this "subitizing" almost as well as we can and probably have their own neurons for it, but their understanding of numbers seems shallower than a primate's.) So: - the artificial neurons are far simpler than cat neurons - the links between the artificial neurons in neuromorphic computers are far simpler than the links between cat neurons in cat brains - even in naive ops/second cats are not slouches Simply matching the number of cat neurons and waving hands about "fast semiconductors" is not enough for me to think this machine is as powerful as a cat's brain. (And even if Intel made a computer theoretically capable of emulating a cat, I suspect it'll take several decades to develop AI software which is sophisticated enough to do the job.)
- abeppu 2y ago
  I agree with much of this, but I think we also need to acknowledge our own ignorance over - how much of the complexity of mammalian neurons is "incidental" complexity, due to also needing to be living, functioning cells, etc as versus "necessary" complexity needed to perform as well as they do - how much specialization between regions and the details of the larger "connectome" are, vs how much can be accomplished by enough repetition of the right local structure (e.g. Hawkins' "thousand brains" etc) > It is reasonable to suppose that the structural complexity of mammal/bird brains is an important part why their cognitive abilities are so much greater than 2024 AI. Statements like this seem a premature to me -- we haven't really had systems with a simple (e.g. repeating) large scale structure and efficient adapting durable-memory-and-compute to really evaluate with, right?