5 ms·
The human brain isn't randomly initialized. It's undergone 500m years of pretraining.
by bhickey 2y ago
The human brain isn't randomly initialized. It's undergone 500m years of pretraining.
- LoganDark 2y agoThis makes me wonder if human brains can be genetically predisposed to a particular dominant language. I'd imagine not since that isn't typically a factor in selection, but I still wonder.
- bhickey 2y agoI doubt it. Language and human evolution operate on different time scales. We wouldn't be able to converse with someone from 13th century England. If anything I will expect selective pressure on languages—those that are easy to use are more likely to be adopted. Secondarily, I would expect this effect to be swamped by other factors (e.g. conquest).
- Grimblewald 2y agoFrom what I have read and have come to understand, it is more that we are generally predisposed to human language in general, specific portions of our brain especially so.
- LoganDark 2y ago> it is more that we are generally predisposed to human language in general I understand, that's not what I was wondering.
- trott 2y ago> The human brain isn't randomly initialized. It's undergone 500m years of pretraining. All of the information accumulated by evolution gets passed through DNA. For humans, that's well under 1GB. Probably a very tiny fraction of that determines how the brain works at the algorithmic level. You should think of this information as the "software" of the brain, not pretrained LLM weights (350GB for GPT-3).