3 ms·
Why can't a small neural network do what LLMs do? Same reason animals can't learn human language. They don't have the capacity. LLMs started getting interestin
by api 3y ago
Why can't a small neural network do what LLMs do? Same reason animals can't learn human language. They don't have the capacity.
LLMs started getting interesting "all the sudden" when they hit a certain scale, just like biological brains.
- bbor 3y agoI mean, that's kinda exactly Chomsky's point, minus the implication (perhaps malinferred) that human-like linguistic is an inevitable or universal characteristic of intelligent minds. There's lots of animals that have very complex brains, and that engage in very complex behaviors - the fact that none of them have even a hint of linguistic ability seems like a strong indicator that these faculties are a human-specific evolution, and not just... well IDK how to even sum up an anti-UG/GG view. "Kids just sorta figure it out" I guess? Although who knows! As Chomsky likes to say, this whole field is in a pre-Galilean state due to the impossibility of conducting comparative studies EDIT: Oh just realized you were the parent comment. Well I'd say the small NN vs. LLM example still doesn't convince me of the likelihood of your statement as I understand it; it goes without saying that lots of animals are much better at intuitive understandings of physics (well, kinetics at least) than humans. You ever seen those snakes that jump from tree to tree? craziest shit you'll ever see