4 ms·
Imagine being told all you need to do to learn Spanish, is to read a 300,000 word Spanish dictionary end to end so that you can probalistically come up with 100
by asdff 3y ago
Imagine being told all you need to do to learn Spanish, is to read a 300,000 word Spanish dictionary end to end so that you can probalistically come up with 1000 conversational phrases. Anyone who has learned a language can tell you it just doesn't work like that. You don't work by accumulating a massive dataset and training on it. No one can hold such a massive dataset of anything in their head at once.
- kristopolous 3y agoGoing from zero to one is different than one to two ("second language acquisition") Zero to one is closer to mimicry and immersion. There's a long Wikipedia article on the field of study https://en.m.wikipedia.org/wiki/Language_acquisition https://en.m.wikipedia.org/wiki/Language_acquisition Furthermore, humans probably aren't static learners and likely have more beneficial times of certain study than others. There's a theory in that too https://en.m.wikipedia.org/wiki/Critical_period_hypothesis https://en.m.wikipedia.org/wiki/Critical_period_hypothesis Saying there's a "digital brain" is more of a framework since the term "brain" looks like it's a moving target In another comment I referred to these systems as like comparing hydraulic pumps to human biceps, cars to horses, etc. We can use the same units of measure, give them the same tasks, but saying they're the same thing only works in the world of poetry
- jackmott42 3y agoThe LLM doesn't hold the whole dataset in its head either.
- aezart 3y agoNot for a lack of trying. If you give it a small enough training set or a big enough neural network, it will directly memorize the whole thing. You have to intentionally make its brain too small to do that in order to force it to find patterns in the data instead.