FWIW I just tried making a maze game and having an LLM solve it step by step, where each move the LLM has to choose from "move forward", "turn left", "turn righ

FWIW I just tried making a maze game and having an LLM solve it step by step, where each move the LLM has to choose from "move forward", "turn left", "turn right", and is provided with a current state e.g., "right: blocked, left: open, forward: open, back: blocked" after each move. I wanted to see if the LLM could maintain some internal context so as not to explore paths that it has already seen before. In chat mode, it didn't. It would get stuck going back and forth along already-explored paths indefinitely. So I tried a "thinking" mode, where after each move, I asked it to summarize what it knew about the maze thus far. Then instead of continuing the chat I'd start a new one, providing the LLM with its previous instance's summary. With gpt4, the summaries were very verbose, and each subsequent iteration would append to it and maybe rephrase a bit. It got very slow, and didn't seem to help a whole lot. They were so verbose, it was hard to tell if they accurately represented the maze or not. "On step N, you went forward, and you found walls on two sides but not the front, so I would recommend trying going forward again for your subsequent move". With o3-mini, the summaries were more concise and more mathematical looking, easy to understand. "Current position: (4,3) orientation: E. History: (0,0): walls N,E; open S,W. (0,1): walls ...". But it would get them wrong sometimes, so didn't help in the long run. I think maybe I could improve the results by playing around with the prompts a bit, but I'm moderately convinced at this point that they're not good enough to maintain an internal state nor externalize a scratchpad state that would allow them to solve a maze one step at a time efficiently. Much less, make sense out of a full RPG.