3 ms·
I think the basic things to take away from the article are: 1. that LLMs are just a massive “what word comes next given your training corpus” algorithm and sca
by usgroup 4y ago
I think the basic things to take away from the article are:
1. that LLMs are just a massive “what word comes next given your training corpus” algorithm and scale has lead to the main effects of interest.
2. Surprisingly many human tasks can be represented by this simple “what word comes next” question given a big enough corpus.
3. But it has nothing to do with how a human executes those tasks because —- for example —- knowing the country south of Rwanda and knowing the likely completion of “the country south of Rwanda is ___” in a corpus, are not the same thing at all.
4. The key reason being that LLM has no access to semantic knowledge beyond correlation in its corpus, but you have access to causality.
5. So it is absurd to compare the LLM to people —- not because it’s like comparing submarines to fish, because in this case they do not both even “swim and live in the sea”.
I think these things have little to do with definitions of “what intelligence is” and the like, and the author is far from a Luddite.