6 ms·

I was expecting the article to mention a fact I just looked up: Nvidia's newest H100 GPU has roughly 80 billion transistors which is roughly equivalent to the n

I was expecting the article to mention a fact I just looked up: Nvidia's newest H100 GPU has roughly 80 billion transistors which is roughly equivalent to the number of neuron cells in the human brain. I would be surprised if transistors are as flexible as human neurons, but it's interesting that we're just getting to the right order of magnitude in terms of quantity assuming they can fill a similar role.

baz00 3y ago
Perhaps I see this as naivety but it feels wrong reading it. A transistor is a really simple functional unit. A neuron is a hugely complex system on its own which is 3 orders of magnitude the size or more and 3d to boot. They are nowhere near functionally equivalent in any way, shape or form. Rationally comparing them is difficult. It's like comparing a light switch to an M3 SoC.
- deleted 3y ago
  [deleted]
- hexaga 3y ago
  It's worth noting that much of the cellular complexity isn't for information processing (insofar as anything can be 'for' something in biological systems). There's all the implicit self-replication stuff, everything you need to tap into energy / physical resources, self-repair, homeostasis, coexistence with other living cells, etc ad nauseum. In my opinion, that's the interesting axis of comparison (the i/p ability w.r.t. data I care about). I don't particularly care that my transistors can't reproduce or repair themselves from an embedded copy of 'the entire human tech-tree, so far'. > 3 orders of magnitude the size or more and 3d to boot To be fair, transistors are orders of magnitude faster in time than neurons. > It's like comparing a light switch to an M3 SoC. Bird to a plane is the analogy I see a lot around here. It feels apt.
  - bognition 3y ago
    The morphology (cellular shape) of a neuron absolutely is involved in the processing of information. Additionally neurons aren’t just simple on/off units like we pretend they are. They are complicated little computers. They change their shape, gene expression, and electrical sensitivity in response to environmental factors, chemical signals from other neurons, and internal clocks. They are insanely complicated. While the analogy of light switch compared to an M3 isn’t exactly accurate it does a good job of signaling that neurons and transistors are very much different thjngs.
    - hexaga 3y ago
      Absolutely. And every time someone talks about what biological systems do, the word 'involved' comes into play. The morphology of neurons is involved in information processing, I don't doubt that at all. It's also involved in staying alive, and staying healthy, and 'being part of the organism', and a great many other things. It is not at all clear which specific aspects of the morphology are 'for' processing information we care about in the context of intelligence (because again, even info processing can be 'for' things like making sure you breathe reliably).
      - profile53 3y ago
        I think the interesting (and scientifically unanswered) question here is: what is the minimum amount of a neuron needed for information processing alone? If we look at the other side, the replication (giant fab facilities) and energy production/management (power lines, PSUs, etc) are separate from the information processing in a computer chip. But, if we consider all of the pieces involved in power generation and replication as part of the chip (just like a brain has), then the complexity of the brain seems loosely equal.
      - robwwilliams 3y ago
        It is generally accepted that the locations and molecular structure of synapses are a critical determinant of CNS function. And a typical neuron will have and make hundreds to several thousands synapses. So the more appropriate unit to compare to CPUs and GPUs is synapse number, not neuron number. Here is an open version of my Annual Review of Neuroscience article with Karl Herrup on the control of neuron number. Old paper but it is one of the first publications to give an accurate estimate of neuron numbers in the human brain right: https://www.nervenet.org/papers/NUMBER_REV_1988.html https://www.nervenet.org/papers/NUMBER_REV_1988.html
  - sroussey 3y ago
    Transistors are just amplifiers. But you can arrange several to get a logical NOR gate and Boolean algebra becomes available at a tiny physical scale. But going from a tiny amplifier large digital systems makes you think: what if we don’t go that direction? It’s so embedded into our thinking of silicon, makes me ponder what other base structures we can do and what other algebras might be more useful today.
beebeepka 3y ago
I mean, I'm not a doctor... or, a roboticist, but 80 billion sounds like a really low number. The brain is physically much bigger than a GPU, even a fat one
- JohnFen 3y ago
  A neuron is also physically much larger than a transistor. But the comparison isn't apt to begin with. It would take quite a few transistors to even approximate the functionality of a neuron.
Traubenfuchs 3y ago
> 80 million neurons With, each, more than 60 different types of neurotransmitters, exocytosis, endocytosis, transporters, destructive enzymes, pre- and postsynaptic receptors, ~7000 connections to other neurons and much more I did not think of, even a single neuron is an unbelievably complex machine all by itself.
- deleted 3y ago
  [deleted]
JohnFen 3y ago
> I would be surprised if transistors are as flexible as human neurons I think the bigger difference is that transistors and neurons are not actually analogous. They operate in completely different ways, using completely different "encodings".
- deleted 3y ago
  [deleted]
dreamcompiler 3y ago
The brain also has around 100 trillion connections. In other words, the average fan-in/fan-out of a real neuron is on the order of 1000. No digital circuit can come close to this because there's not enough real estate on a 2D chip to route the connections (although chips are moving into 3D). For now, that kind of fan-in/fan-out has to be simulated sequentially. https://www.scientificamerican.com/article/100-trillion-connections/ https://www.scientificamerican.com/article/100-trillion-conn...
- hliyan 3y ago
  Also, it's massively, asynchronously parallel, in that it's not driven by a central clock signal.
  - robwwilliams 3y ago
    And that is a huge mystery!
mr_toad 3y ago
A transistor has only one base that controls the output. It’d be like a brain cell with only one input and one output. A typical brain cell can have thousands of inputs and many outputs. The best GPUs are still many orders of magnitude less complex than the human brain. However I think we are getting somewhere. We’re getting to the point where artificial neural networks can rival the complexity of simple invertebrates.
- deleted 3y ago
  [deleted]

baz00 3y ago

Perhaps I see this as naivety but it feels wrong reading it. A transistor is a really simple functional unit. A neuron is a hugely complex system on its own which is 3 orders of magnitude the size or more and 3d to boot. They are nowhere near functionally equivalent in any way, shape or form. Rationally comparing them is difficult. It's like comparing a light switch to an M3 SoC.

deleted 3y ago
[deleted]
hexaga 3y ago
It's worth noting that much of the cellular complexity isn't for information processing (insofar as anything can be 'for' something in biological systems). There's all the implicit self-replication stuff, everything you need to tap into energy / physical resources, self-repair, homeostasis, coexistence with other living cells, etc ad nauseum. In my opinion, that's the interesting axis of comparison (the i/p ability w.r.t. data I care about). I don't particularly care that my transistors can't reproduce or repair themselves from an embedded copy of 'the entire human tech-tree, so far'. > 3 orders of magnitude the size or more and 3d to boot To be fair, transistors are orders of magnitude faster in time than neurons. > It's like comparing a light switch to an M3 SoC. Bird to a plane is the analogy I see a lot around here. It feels apt.
- bognition 3y ago
  The morphology (cellular shape) of a neuron absolutely is involved in the processing of information. Additionally neurons aren’t just simple on/off units like we pretend they are. They are complicated little computers. They change their shape, gene expression, and electrical sensitivity in response to environmental factors, chemical signals from other neurons, and internal clocks. They are insanely complicated. While the analogy of light switch compared to an M3 isn’t exactly accurate it does a good job of signaling that neurons and transistors are very much different thjngs.
  - hexaga 3y ago
    Absolutely. And every time someone talks about what biological systems do, the word 'involved' comes into play. The morphology of neurons is involved in information processing, I don't doubt that at all. It's also involved in staying alive, and staying healthy, and 'being part of the organism', and a great many other things. It is not at all clear which specific aspects of the morphology are 'for' processing information we care about in the context of intelligence (because again, even info processing can be 'for' things like making sure you breathe reliably).
    - profile53 3y ago
      I think the interesting (and scientifically unanswered) question here is: what is the minimum amount of a neuron needed for information processing alone? If we look at the other side, the replication (giant fab facilities) and energy production/management (power lines, PSUs, etc) are separate from the information processing in a computer chip. But, if we consider all of the pieces involved in power generation and replication as part of the chip (just like a brain has), then the complexity of the brain seems loosely equal.
    - robwwilliams 3y ago
      It is generally accepted that the locations and molecular structure of synapses are a critical determinant of CNS function. And a typical neuron will have and make hundreds to several thousands synapses. So the more appropriate unit to compare to CPUs and GPUs is synapse number, not neuron number. Here is an open version of my Annual Review of Neuroscience article with Karl Herrup on the control of neuron number. Old paper but it is one of the first publications to give an accurate estimate of neuron numbers in the human brain right: https://www.nervenet.org/papers/NUMBER_REV_1988.html https://www.nervenet.org/papers/NUMBER_REV_1988.html
- sroussey 3y ago
  Transistors are just amplifiers. But you can arrange several to get a logical NOR gate and Boolean algebra becomes available at a tiny physical scale. But going from a tiny amplifier large digital systems makes you think: what if we don’t go that direction? It’s so embedded into our thinking of silicon, makes me ponder what other base structures we can do and what other algebras might be more useful today.

beebeepka 3y ago

I mean, I'm not a doctor... or, a roboticist, but 80 billion sounds like a really low number. The brain is physically much bigger than a GPU, even a fat one

JohnFen 3y ago
A neuron is also physically much larger than a transistor. But the comparison isn't apt to begin with. It would take quite a few transistors to even approximate the functionality of a neuron.

Traubenfuchs 3y ago

> 80 million neurons With, each, more than 60 different types of neurotransmitters, exocytosis, endocytosis, transporters, destructive enzymes, pre- and postsynaptic receptors, ~7000 connections to other neurons and much more I did not think of, even a single neuron is an unbelievably complex machine all by itself.

deleted 3y ago
[deleted]

JohnFen 3y ago

> I would be surprised if transistors are as flexible as human neurons I think the bigger difference is that transistors and neurons are not actually analogous. They operate in completely different ways, using completely different "encodings".

deleted 3y ago
[deleted]

dreamcompiler 3y ago

The brain also has around 100 trillion connections. In other words, the average fan-in/fan-out of a real neuron is on the order of 1000. No digital circuit can come close to this because there's not enough real estate on a 2D chip to route the connections (although chips are moving into 3D). For now, that kind of fan-in/fan-out has to be simulated sequentially. https://www.scientificamerican.com/article/100-trillion-connections/ https://www.scientificamerican.com/article/100-trillion-conn...

hliyan 3y ago
Also, it's massively, asynchronously parallel, in that it's not driven by a central clock signal.
- robwwilliams 3y ago
  And that is a huge mystery!

mr_toad 3y ago

A transistor has only one base that controls the output. It’d be like a brain cell with only one input and one output. A typical brain cell can have thousands of inputs and many outputs. The best GPUs are still many orders of magnitude less complex than the human brain. However I think we are getting somewhere. We’re getting to the point where artificial neural networks can rival the complexity of simple invertebrates.

deleted 3y ago
[deleted]