People have tried this! And for not-complex problems, the newer models do give the right answer and yet usually fail on the complex tasks. Prompting does help

People have tried this! And for not-complex problems, the newer models do give the right answer and yet usually fail on the complex tasks. Prompting does help though - if you ask the model to check it's own answer, it can often catch errors it made and improve. But the interesting part is precisely how does the model succeed at all at any spatial reasoning task? Surely it hasn't seen all spatial reasoning tasks in its training set. So, what's the other hypothesis? As you say, people suggest that there could be a world model that is constructed internally by the LLM. However, as you can see right here in this thread that people disagree that you need a world model for solving such tasks.