Part of the breakthrough with LLMs has precisely been about encoding nothing at all about language and letting the transformer figure it out from scratch. We d

Part of the breakthrough with LLMs has precisely been about encoding nothing at all about language and letting the transformer figure it out from scratch. We don't know what is necessary for learning language.