No this is like saying that whovever writes a piece of software understands how it works. Unless one forgot about it or stumbled upon it out of sheer luck. And

No this is like saying that whovever writes a piece of software understands how it works. Unless one forgot about it or stumbled upon it out of sheer luck. And neither of those are the case with LLMs.