Perhaps. But the point is that some prompt will coax it into giving good answers that really make it win the game, if it has a good "world model" of how the gam

Perhaps. But the point is that some prompt will coax it into giving good answers that really make it win the game, if it has a good "world model" of how the game works. And there's no reason to think a language model cannot have such a world model. What exactly that prompt might be, the prompt engineers know best.