6 ms·

Are neurons really more efficient at computing than transistors ?

by fvdessen 3y ago

marricks 3y ago
I imagine that’s a massive “depends on what you want them to do” Add numbers, no of course not, write a book though…
- zmgsabst 3y ago
  Is it true even in that example? 251MJ is what 2000kcal diet for a month works out to. That’s also 2.5kW for 100 servers for 1000 seconds. Is it true that a human would write a better book in a month than a well-designed algorithm? Maybe.
  - marricks 3y ago
    If you want to go with imaginary algorithms that don’t exist yet (or possibly ever) sure such an imaginary computer or system could do it more efficiently.
  - bmitc 3y ago
    What would design the algorithm?
  - Teever 3y ago
    Humans can't eat just anything though, and there's a tremendous amount of energy embodied in human food, including growing, storing, transporting and cookihg it.
    - Engineering-MD 3y ago
      And is also true for computers in different ways. Human food comes included in the natural Environment, electricity requires some degree of commercial mining.
  - vasco 3y ago
    > Is it true that a human would write a better book in a month than a well-designed algorithm? The "well designed" plays a big role in this sentence. At some point you're just writing the book with a different "pen".
glenstein 3y ago
My understanding is that neurons are simultaneously electrically active and chemically active, and are organized into larger systems that are dynamically plastic. Somewhere in there there's some efficiencies being achieved that come from being functionally active in ways that it don't make for an easy one to one comparison to transistors.
- 082349872349872 3y ago
  According to https://www.rand.org/content/dam/rand/pubs/research_memoranda/2008/RM704.pdf https://www.rand.org/content/dam/rand/pubs/research_memorand... (1951) neurons (and, we know, transistors, also) wind up mapping onto (the same) basic finite automata mathematics.
  - glenstein 3y ago
    Right, lots of things can map onto models of computation described in typical theory of computation textbook, and that gives you your one to one comparison to transistors. You can also find equivalences between board games, lego, conways game of life etc. etc. But what I'm saying is, there is a layer (or layers) of organic function that is abstracted away in machine learning/AI. While the field does borrow ideas liberally from brain architecture (as it well should), you nevertheless have neurons doing things like chemical signalling, metabolic regulation, unique site-specific structural organization, and processes for dynamic reorganization of structure as needed. That is, not just in the informational representation, which can be sufficiently abstract to model practically anything, but in the hardware itself upon which the information rests. It's there that I suspect there are efficiencies gained by brains over transistors.
nicklecompte 3y ago
Neurons aren't anything like transistors: they're CPU cores, and each individual neuron in a mammalian cerebral cortex is the equivalent of a deep artificial neural network: https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... In particular the human brain is an 80-billion core processor with each core running at 300Hz. Its architecture is not so good at FLOPs and other things we typically use computers for, but overall human brains dramatically outperform anything in 2024 except the world's fastest supercomputer.
- spiderice 3y ago
  > overall Meaning at specific tasks, right? Because obviously even the weakest modern computers out perform humans at certain kinds of tasks.
  - nicklecompte 3y ago
    No, I mean in terms of raw computational power: 300Hz * 80 billion cores. Again the architecture does not work well for churning through a giant array of floating point arithmetic since 300Hz is a limiting factor for that specific type of computation. But that's not the kind of problem the human brain was designed to solve.
- wruza 3y ago
  I think the correct term is “outclasses”, not “outperforms”. Our performance is pretty bad, especially on average and compared to other species (when comparable).
- usrbinbash 3y ago
  > they're CPU cores, No, they are really not. A Neuron on its own is capable of exactly one thing, and that's signal integration from it's dendrites. That's it. Wait until the excitation potential reaches the point where the Potassium channels switch and boum: Signal cascade goes down the axon. The only moderation a single neuron is capable of, is varying signal resistance to filter out repeating noise. > https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... Computational complexity doesn't indicate computational capability. There are any number of systems that stubbornly refuse to be capable supercomputers, despite being incredibly hard to simulate. Accurately simulating every aspect of the liquid flow through an acorn leaf, with it's billions of micro and nanometer channels, would easily be one of the most difficult tasks in the history of computing, and yet, acorn leaves are not even capable of the simplest thought, and the entire modus operandi of water flow through an entire tree can be accurately summarized in a simple 1-page diagram. Every single muscle fiber with its trillions of moving cytoskeleton components, has a much higher complexity than a neuron, that doesn't mean muscles are supercomputers.

Are neurons really more efficient at computing than transistors ?

marricks 3y ago

I imagine that’s a massive “depends on what you want them to do” Add numbers, no of course not, write a book though…

zmgsabst 3y ago
Is it true even in that example? 251MJ is what 2000kcal diet for a month works out to. That’s also 2.5kW for 100 servers for 1000 seconds. Is it true that a human would write a better book in a month than a well-designed algorithm? Maybe.
- marricks 3y ago
  If you want to go with imaginary algorithms that don’t exist yet (or possibly ever) sure such an imaginary computer or system could do it more efficiently.
- bmitc 3y ago
  What would design the algorithm?
- Teever 3y ago
  Humans can't eat just anything though, and there's a tremendous amount of energy embodied in human food, including growing, storing, transporting and cookihg it.
  - Engineering-MD 3y ago
    And is also true for computers in different ways. Human food comes included in the natural Environment, electricity requires some degree of commercial mining.
- vasco 3y ago
  > Is it true that a human would write a better book in a month than a well-designed algorithm? The "well designed" plays a big role in this sentence. At some point you're just writing the book with a different "pen".

glenstein 3y ago

My understanding is that neurons are simultaneously electrically active and chemically active, and are organized into larger systems that are dynamically plastic. Somewhere in there there's some efficiencies being achieved that come from being functionally active in ways that it don't make for an easy one to one comparison to transistors.

082349872349872 3y ago
According to https://www.rand.org/content/dam/rand/pubs/research_memoranda/2008/RM704.pdf https://www.rand.org/content/dam/rand/pubs/research_memorand... (1951) neurons (and, we know, transistors, also) wind up mapping onto (the same) basic finite automata mathematics.
- glenstein 3y ago
  Right, lots of things can map onto models of computation described in typical theory of computation textbook, and that gives you your one to one comparison to transistors. You can also find equivalences between board games, lego, conways game of life etc. etc. But what I'm saying is, there is a layer (or layers) of organic function that is abstracted away in machine learning/AI. While the field does borrow ideas liberally from brain architecture (as it well should), you nevertheless have neurons doing things like chemical signalling, metabolic regulation, unique site-specific structural organization, and processes for dynamic reorganization of structure as needed. That is, not just in the informational representation, which can be sufficiently abstract to model practically anything, but in the hardware itself upon which the information rests. It's there that I suspect there are efficiencies gained by brains over transistors.

nicklecompte 3y ago

Neurons aren't anything like transistors: they're CPU cores, and each individual neuron in a mammalian cerebral cortex is the equivalent of a deep artificial neural network: https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... In particular the human brain is an 80-billion core processor with each core running at 300Hz. Its architecture is not so good at FLOPs and other things we typically use computers for, but overall human brains dramatically outperform anything in 2024 except the world's fastest supercomputer.

spiderice 3y ago
> overall Meaning at specific tasks, right? Because obviously even the weakest modern computers out perform humans at certain kinds of tasks.
- nicklecompte 3y ago
  No, I mean in terms of raw computational power: 300Hz * 80 billion cores. Again the architecture does not work well for churning through a giant array of floating point arithmetic since 300Hz is a limiting factor for that specific type of computation. But that's not the kind of problem the human brain was designed to solve.
wruza 3y ago
I think the correct term is “outclasses”, not “outperforms”. Our performance is pretty bad, especially on average and compared to other species (when comparable).
usrbinbash 3y ago
> they're CPU cores, No, they are really not. A Neuron on its own is capable of exactly one thing, and that's signal integration from it's dendrites. That's it. Wait until the excitation potential reaches the point where the Potassium channels switch and boum: Signal cascade goes down the axon. The only moderation a single neuron is capable of, is varying signal resistance to filter out repeating noise. > https://www.quantamagazine.org/how-computationally-complex-is-a-single-neuron-20210902/ https://www.quantamagazine.org/how-computationally-complex-i... Computational complexity doesn't indicate computational capability. There are any number of systems that stubbornly refuse to be capable supercomputers, despite being incredibly hard to simulate. Accurately simulating every aspect of the liquid flow through an acorn leaf, with it's billions of micro and nanometer channels, would easily be one of the most difficult tasks in the history of computing, and yet, acorn leaves are not even capable of the simplest thought, and the entire modus operandi of water flow through an entire tree can be accurately summarized in a simple 1-page diagram. Every single muscle fiber with its trillions of moving cytoskeleton components, has a much higher complexity than a neuron, that doesn't mean muscles are supercomputers.