Language is indeed communicated abstraction of concepts, but it emerged under a lot of constraints (our auditory system, our brains inherent bias towards visual

Language is indeed communicated abstraction of concepts, but it emerged under a lot of constraints (our auditory system, our brains inherent bias towards visual stimuli etc. ). Predicting in this constrained system most likely is suboptimal. Imagine translating language into an optimized representation free from human constraints, doing autoregressive prediction in this domain and only than translate back. As far as I understand current models, this is not yet how they work.