I wonder if it's not the LLM making the breakthrough but rather that the person using the system just needed the information available presented in a clear and

I wonder if it's not the LLM making the breakthrough but rather that the person using the system just needed the information available presented in a clear and orderly fashion to make the breakthrough itself. After all, the LLM currently has no cognizance, it is unable to understand what it is saying in a meaningful way. At its best it is a P-Noid Zombie machine, right? In my opinion anything amazing that comes from an LLM only becomes amazing when someone who was capable of recognizing the amazingness perceives it, like a rewrite of a zen koan, "If an LLM generates a new work of William Shakespeare, and nobody ever reads it, was anything of value lost?"