I don't need to prove anything. We know for a fact that LLMs are stochastic parrots. That's why they confabulate, if they were "intelligent" they wouldn't just

I don't need to prove anything. We know for a fact that LLMs are stochastic parrots. That's why they confabulate, if they were "intelligent" they wouldn't just pass off nonsense as fact. They have been fed incredible amounts of data which allows them to stochastically parrot really complex things, this makes people think they're intelligent. Designing a test that actually tests intelligence rather than parroting ability is tricky, but whenever people do they fail them. I just logged into ChatGPT and asked it repeatedly to generate a novel thought experiment intended to prove that LLMs are incapable of reasoning. I kept trying to refine it by explaining what was wrong with each suggestion. It completely failed. It just spat out variations of well known dilemmas or paradoxes, even when I specifically asked it to create something original it just made nonsense.