They do, but it's a form of imitation, not actually knowing what they don't know. Ask an LLM to imitate a confident physicist and it will try, regardless of ho

They do, but it's a form of imitation, not actually knowing what they don't know. Ask an LLM to imitate a confident physicist and it will try, regardless of how much physics it knows. Or if you tell ChatGPT that it's wrong multiple times, it may learn the pattern and assume it's always wrong, resulting in a downward spiral. (This can happen when using Code Interpreter and it makes several failed attempts to correct a mistake.) The difficult research problem is training it to have an accurate model of what it knows.