To apply the scientific method and prove correctness, you need to define the subject first. Meaning is ill-defined, just like consciousness, intelligence, and o

To apply the scientific method and prove correctness, you need to define the subject first. Meaning is ill-defined, just like consciousness, intelligence, and others. LLMs can clearly grasp higher order abstractions and concepts just by reordering the words. In fact, embeddings (developed much earlier) are specifically intended to represent the semantical meaning extracted from text using just statistical methods; until the introduction of transformers they lacked a good architecture to demonstrate the usefulness of that. This makes many people argue that being a sentence builder is enough to be intelligent, as they also received most of their intelligence from the same source (concentrated experience of someone else - social intelligence). "Honesty" and other forms of self-introspection is just a high level construct which current models aren't trained for, and likely not a fundamental issue with being a stochastic parrot.