4 ms·
I’m surprised to see someone of Carmack’s caliber making this mistake. If you’ve exposed a gorilla or chimp brain to the same amount of language content they st
by dwallin 4y ago
I’m surprised to see someone of Carmack’s caliber making this mistake. If you’ve exposed a gorilla or chimp brain to the same amount of language content they still wouldn’t be able to converse like a human can. Meanwhile almost every human can utilize spoken language, despite huge variations in their training data. This implies that most of the language capabilities of the brain depend on structural elements that have been evolutionarily trained and not on the fine-tuning that a specific human achieves in their lifetime. Think of how many billions to trillions of attempts evolution had. And it was building off of models already trained (with many orders of magnitude more iterations) at the tasks of audio processing and sound recognition.
Then add in the fact that we are only talking about language acquisition and all indications are that sufficiently large models (And the human brain is quite the large model) have significant benefits from cross training and it starts to become clear how false of a premise this is. 11 million bits of neural information are being pumped to our brains every second of our lives. That’s 80 gb of external data ingested during our waking hours in a single day plus who knows what sorts of internal synthetically generated data, all of this just for fine-tuning.