3 ms·
The Chomsky hierarchy will remain unchanged and very useful for constructed languages. I didn't mean break out, as in create a higher level. LLMs are not goin
by vimax 3y ago
The Chomsky hierarchy will remain unchanged and very useful for constructed languages. I didn't mean break out, as in create a higher level. LLMs are not going to magically become more than a turing machine or anything. I meant we need to stop thinking about spoken language in terms of the hierarchy and try to find another approach.
You can't tell me with a straight face that actual spoken language fits well in that hierarchy. It seems we can shoehorn it in with a 100B parameter turing machine definition, but that just makes my point that the constructed language hierarchy is not well suited for describing spoken languages if it takes such a large definition.
We should be able to find something like a new probabilistic theory of language that will do a much better job of describing spoken languages. Such a theory could help explain some LLM behaviors we see, and apply generally to other forms of noisy communication.
- visarga 3y ago> 100B parameter ... not well suited for describing spoken languages if it takes such a large definition. Humans by contrast use about 700T synapses to support our own understanding of language.