3 ms·
A little nitpick; a biological neuron is much more complex than it's ml-model equivalent. a simple weighted function cannot fully replicate a neuron. That's wh
by legacynl 2y ago
A little nitpick; a biological neuron is much more complex than it's ml-model equivalent. a simple weighted function cannot fully replicate a neuron.
That's why it's almost certain that a biological brain with a billion synapses outperforms a model with a billion parameters.
- mort96 2y agoIsn't that what they meant by this? > the comparison is not apples-to-apples, because each synapse is much more complex than a single parameter in a weight matrix.
- daveguy 2y agoIt isn't just "not apples to apples". It's apples to supercomputers.
- legacynl 2y agowell yeah but it's un-obviously a very big difference that basically invalidates any conclusion that you can make with this comparison.
- mort96 2y agoI don't think so: it seems reasonable to assume that biological neurons are strictly more powerful than "neural network" weights, so the fact that a human brain has 3 orders of magnitude more biological neurons than language models have weights tells that we should expect, as an extreme lower bound, 3 orders of magnitude difference.
- joe_the_user 2y agoIt's not a "nitpick", it's a complete refutation. LLM don't have a strong relationship to brains, they're just math/computer constructs.
- Purplehermann 2y agoThe brain is insanely energy efficient, this is not the same as intelligence efficient