I wonder how far it is from this to generating language reasoning about the game from the game itself, rather than learning a large corpus of language, like LLM

I wonder how far it is from this to generating language reasoning about the game from the game itself, rather than learning a large corpus of language, like LLMs do. That would be a true grounded language generator