This is the most disturbing thing about LLMs, imo. Not that "LLMs aren't intelligent, they just say something plausible and if they don't know what they're talk

This is the most disturbing thing about LLMs, imo. Not that "LLMs aren't intelligent, they just say something plausible and if they don't know what they're talking about they'll fluently bs about it" but that "when forced to respond to a question out of their depth humans sound exactly like LLMs... oh no humans aren't intelligent" and in fact we really don't have a solid definition of intelligence beyond "yeah but did it work though?" The only way to distinguish genius and madness, the old saying goes, is in the results.