3 ms·

Do you think a single neuron has more compute than a H100? I'm not sure even energy based theoretical bounds on computation power allow this even in principle.

morpheos137 2y ago
What do you even mean? You are comparing apples and oranges. Can an H100 churn through deterministic instructions to get to a fixed result in a computer's combinatorial envelope fast? No doubt. However an abacus does not know how or what to add to achieve a goal. Can an H100 or a bunch of H100s achieve better biological control than the human brain in real time as a self directed agent? No way. Apples and oranges.
- Davidzheng 2y ago
  No i mean a neuron has input electrical signals and outputs. Do you think I cannot simulate a single neuron to fidelity on a H100? Probably theoretically it is possible that it cannot if it's leveraging some physics to do optimization but I think in practice it's possible. Fwiw I think 10e6 h100s can act as a competent self directed agent with the right algorithms yes
  - Retric 2y ago
    I think 1e6 h100s could run something that’s intelligent enough to hold a meaningful conversation as in AGI assuming we figured out the appropriate algorithms and training methods. But your description of Neurons is off. They output chemical signals, which is why painkillers can dull the signal etc. Many drugs bind to receptors on neurons which then block signals which wouldn’t work if these where electrical connections. The do respond to electrical stimulation, and it’s often said they use electrical signals to carry information along the body of the neuron but the actual signal only moves up to ~120 m/s a long way from how fast electrical signals move. https://en.wikipedia.org/wiki/Nerve_conduction_velocity https://en.wikipedia.org/wiki/Nerve_conduction_velocity As to simulation of a neuron, the brains got 86 billion neurons and the peripheral nervous system is doing useful work here. A full fidelity simulation of even one neuron is completely off the table simply because it’s an analog system where timing matters to an arbitrary precision, but what’s good enough is really just a question of context. Good enough to test new drugs for depression? No, though good enough to drive a car is a reasonable goal.
Retric 2y ago
Yes. It’s allowed because the energy limits in the Landauer's principle are based on irreversible state changes. When an Antibody doesn’t match a surface protein there’s computation with zero energy loss because there’s no change in state. It amounts to a “free” if statement as long as the result is false and it’s almost always false. Though in a larger context it’s really inefficient as it’s a random process only effective because of how many Antibodies flood the body and how quickly they bounce around and can preform this computation. Cells heavily leverage this kind of computation to the point where it’s not generally thought of as computation until something happens.
- Davidzheng 2y ago
  Thanks for the explanation--that was helpful. However I am not sure if microscopic states of the neuron are propagated as signals to nearby cells or only some sort of macroscopic state (like the sort of abstraction of thermodynamics) and I suspect there are bounds to the speed of change and number of these macroscopic states. I guess at some point we should only count computations above some abstraction level otherwise we also undercount the computation power of a gpu. Are we broadly in agreement? Even if we disagree above the relative computation power of the two

morpheos137 2y ago

What do you even mean? You are comparing apples and oranges. Can an H100 churn through deterministic instructions to get to a fixed result in a computer's combinatorial envelope fast? No doubt. However an abacus does not know how or what to add to achieve a goal. Can an H100 or a bunch of H100s achieve better biological control than the human brain in real time as a self directed agent? No way. Apples and oranges.

Davidzheng 2y ago
No i mean a neuron has input electrical signals and outputs. Do you think I cannot simulate a single neuron to fidelity on a H100? Probably theoretically it is possible that it cannot if it's leveraging some physics to do optimization but I think in practice it's possible. Fwiw I think 10e6 h100s can act as a competent self directed agent with the right algorithms yes
- Retric 2y ago
  I think 1e6 h100s could run something that’s intelligent enough to hold a meaningful conversation as in AGI assuming we figured out the appropriate algorithms and training methods. But your description of Neurons is off. They output chemical signals, which is why painkillers can dull the signal etc. Many drugs bind to receptors on neurons which then block signals which wouldn’t work if these where electrical connections. The do respond to electrical stimulation, and it’s often said they use electrical signals to carry information along the body of the neuron but the actual signal only moves up to ~120 m/s a long way from how fast electrical signals move. https://en.wikipedia.org/wiki/Nerve_conduction_velocity https://en.wikipedia.org/wiki/Nerve_conduction_velocity As to simulation of a neuron, the brains got 86 billion neurons and the peripheral nervous system is doing useful work here. A full fidelity simulation of even one neuron is completely off the table simply because it’s an analog system where timing matters to an arbitrary precision, but what’s good enough is really just a question of context. Good enough to test new drugs for depression? No, though good enough to drive a car is a reasonable goal.

Retric 2y ago

Yes. It’s allowed because the energy limits in the Landauer's principle are based on irreversible state changes. When an Antibody doesn’t match a surface protein there’s computation with zero energy loss because there’s no change in state. It amounts to a “free” if statement as long as the result is false and it’s almost always false. Though in a larger context it’s really inefficient as it’s a random process only effective because of how many Antibodies flood the body and how quickly they bounce around and can preform this computation. Cells heavily leverage this kind of computation to the point where it’s not generally thought of as computation until something happens.

Davidzheng 2y ago
Thanks for the explanation--that was helpful. However I am not sure if microscopic states of the neuron are propagated as signals to nearby cells or only some sort of macroscopic state (like the sort of abstraction of thermodynamics) and I suspect there are bounds to the speed of change and number of these macroscopic states. I guess at some point we should only count computations above some abstraction level otherwise we also undercount the computation power of a gpu. Are we broadly in agreement? Even if we disagree above the relative computation power of the two