When I talk to 8B models, it's often painfully clear that they are operating mostly (entirely?) on the level of language. They often say things that make no sen

When I talk to 8B models, it's often painfully clear that they are operating mostly (entirely?) on the level of language. They often say things that make no sense except from a "word association" perspective. With bigger (400B models) that's not so clear. It would be silly to say that a fruit fly has the same thoughts as me, only a million times smaller quantitatively. I imagine the same thing is true (genuine qualitative leaps) in the 8B -> 400B direction.