I fully agree that context the semantic content itself primes the listenerrs expectations of the next rabbit. But that sort of language model need not be GPT-le

I fully agree that context the semantic content itself primes the listenerrs expectations of the next rabbit. But that sort of language model need not be GPT-level good. It just needs to identify which words/letters are unlikely to be next to each other and which ones they might be mistaken for. An unsophisticated Markov chain could do that. Is this really a 2B parameter problem, or do we do things this way because parameters are currently cheaper than expertise?