> then LLMs simply lack training data to observe those limitations. How would training data add an introspective step to the LLM? At best you can add a lot of

> then LLMs simply lack training data to observe those limitations. How would training data add an introspective step to the LLM? At best you can add a lot of common LLM fitted strategies developed by humans to the training set and thus make the LLM solve a few more problems well, but that will always be hampered by the need for humans to develop those. It is basically just programming the LLM. My view is that LLM is results in natural language programming. Which is great, but it isn't any more intelligence than normal programming. Its just that we already have a lot of cool programs already written in natural languages, and now we get to run those for the first time.