This is utterly unsurprising, since LLMs are statistical models of text, not inference engines (or similar). > Interestingly, the models often produce the corr

This is utterly unsurprising, since LLMs are statistical models of text, not inference engines (or similar). > Interestingly, the models often produce the correct answer initially, but switch to an incorrect response after self-correction. The training corpus almost certainly contains examples that are similar to both cases ("my answer was correct" and "oops, my answer was wrong"). It'll be a coin flip which set the input more closely matches.