4 ms·
> No Language model is yet even close to the scale of the human brain If GPT-4 has 100 trillion parameters, it has as many parameters as the human brain has sy
by hwillis 3y ago
> No Language model is yet even close to the scale of the human brain
If GPT-4 has 100 trillion parameters, it has as many parameters as the human brain has synapses. Synapses are a lot simpler than parameters; they're digital. A single neuron needs many synapses, all of roughly equal weight, emitting many pulses over a short time in order to convey a single weighted value.
On top of that, you may have heard that the human brain does a lot of things besides writing. You subtract the motor cortex, the visual and limbic systems etc... a 100 trillion parameter model is unambiguously larger than the language processing portions of the human brain.
> Brains don't start at 0. Evolution, dna/rna etc. There's obviously some pre disposition for language learning in humans but that alone isn't enough ground for a "universal grammar"
The human genome is 24 gigabits long. It's negligibly small compared to a language model.
- opportune 3y agoSynapses, and the brain in general, have more degrees of freedom than just a single floating point parameter allows. There are multiple neurotransmitters floating within the synapse, each side of the synapse has different numbers/density of various receptors, a bunch of proteins and such are constantly being created and used by each cell to change the brain structure itself. It’s not just a digital signal
- dahart 3y agoGoogle says gpt 4 has ~1 trillion parameters.
- famouswaffles 3y ago>If GPT-4 has 100 trillion parameters It doesn't. >Synapses are a lot simpler than parameters Not true but someone else has explained why. >a 100 trillion parameter model is unambiguously larger than the language processing portions of the human brain. GPT-4 is not that big. The technology to run such a model at scale is simply not feasible yet. >The human genome is 24 gigabits long. It's negligibly small compared to a language model. I'm sorry bit this makes no sense. How many gigabits long the genome is has no bearing on how impactful it is in steering the development of the human brain in comparison to a ML model.
- swexbe 3y agoThere's a single medium.com grifter claiming 100 trillion with nothing to back it up. 1 trillion for the unavailable instruct model and considerably less for the chat model is most likely.