2 ms·
> Our own brains are hardwired for language acquisition by evolution, so we can few-shot it when learning and get there much faster No, we can’t few shot it an
by dragonwriter 2y ago
> Our own brains are hardwired for language acquisition by evolution, so we can few-shot it when learning and get there much faster
No, we can’t few shot it and we don't get there faster (but we develop a lot of other capabilities on the way.) We train on a lot more data; the human brain, unlike an LLM, is training on all that data in processes for ”inference”, and it receives sensory data estimated on the order of a billion bits per second, which means by the time we start using language we’ve trained on a lot of data (the 15 trillion tokens from a ~17 bit token vocabulary that Llama3 is something like the size of a few days of human sense data.) Humans just are trained on and process vastly richer multimodal data instead of text streams.
- int_19h 2y agoI was talking about language acquisition specifically. Most of the data that you reference is visual input and other body sensations that aren't directly related to that. OTOH humans don't take all that much text to learn to read and write.
- dragonwriter 2y ago> I was talking about language acquisition specifically. Yeah, humans don't acquire language separately from other experience. > Most of the data that you reference is visual input and other body sensations that aren't directly related to that. Visual input and other body sensations are not unrelated to language acquisition. > OTOH humans don't take all that much text to learn to read and write. That generally occurs well after they have acquired both language and recognizing and using symbolic visual communication, and they usually have considerable other input in learning how to read and write besides text they are presented with (e.g., someone else reading words out loud to them.)